Camera System Using Stabilizing Gimbal

ABSTRACT

Disclosed is an electronic gimbal with camera and mounting configuration. The gimbal can include an inertial measurement unit which can sense the orientation of the camera and three electronic motors which can manipulate the orientation of the camera. The gimbal can be removably coupled to a variety of mount platforms, such as an aerial vehicle, a handheld grip, or a rotating platform. Moreover, a camera can be removably coupled to the gimbal and can be held in a removable camera frame. Also disclosed is a system for allowing the platform, to which the gimbal is mounted, to control settings of the camera or to trigger actions on the camera, such as taking a picture, or initiating the recording of a video. The gimbal can also provide a connection between the camera and the mount platform, such that the mount platform receives images and video content from the camera.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.15/841,555, filed on Dec. 14, 2017, which is a continuation of U.S.patent application Ser. No. 15/307,331 entered on Oct. 27, 2016, nowU.S. Pat. No. 9,874,308, which was filed as 35 U.S.C. 371 National PhaseApplication of International Application No. PCT/US2016/028518 filed onApr. 20, 2016, which claims the benefit of U.S. Provisional PatentApplication No. 62/167,241 filed on May 27, 2015, U.S. ProvisionalPatent Application No. 62/249,879 filed on Nov. 2, 2015, and U.S.Provisional Patent Application No. 62/302,170 filed on Mar. 2, 2016, allof which the contents are incorporated by reference herein.

BACKGROUND Field of Art

The disclosure generally relates to the field of camera gimbals and inparticular a detachable gimbal that can be connected to a camera and toa variety of mount platforms.

Description of Art

Unstabilized videos taken while flying an aerial vehicle or while movingaround at ground level are often so shaky and unstable that they areunusable, not sharable, and unwatchable. The use of an electronic gimbalto stabilize or to set the orientation of a camera is known. A gimbalcan be mounted to a platform such as an electronic vehicle. For example,a camera can be mounted via a gimbal to a remote control road vehicle oraerial vehicle to capture images as the vehicle is controlled remotelyby a user. A gimbal can allow the recording of stable video even whenthe platform is unstable.

However, existing stabilization equipment is large, not portable,expensive and can only be used for stabilization. Moreover, most cameragimbals mounted on remote controlled vehicles do not take into aconsideration a multitude of issues involving the camera itself inrelation to the platform to which it is mounted. These issues include,for example, allowing for a multiplicity of different cameras withdifferent weights and form factors to be mounted to the gimbal, using asecuring mechanism that will allow the gimbal to connect to a variety ofplatforms, preventing or minimizing obstruction of the field of view ofthe camera by components of the gimbal, allowing communication betweenthe platform and the mounted camera, stabilizing video captured by thecamera, and accounting for rotations of the mount platform.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed embodiments have advantages and features which will bemore readily apparent from the detailed description, the appendedclaims, and the accompanying figures (or drawings). A brief introductionof the figures is below.

FIG. 1 is a functional block diagram illustrating an exampleconfiguration of a camera mounted on a gimbal which is, in turn, mountedto a mount platform.

FIG. 2 illustrates an example of a gimbal coupled to a remote controlledaerial vehicle.

FIGS. 3A and 3B illustrate an example of a gimbal and camera.

FIG. 4 illustrates a block diagram of an example camera architecture.

FIG. 5 illustrates an embodiment of a detachable camera frame.

FIG. 6 illustrates a handheld grip coupled to a gimbal and camera.

FIG. 7 illustrates an example configuration of remote controlled aerialvehicle in communication with a remote controller.

FIGS. 8A and 8B illustrates an example of a dampening connection forcoupling a gimbal to a mount platform.

FIG. 9 illustrates an example of a gimbal coupled to a rotatingplatform.

FIGS. 10A, 10B, 10C, and 10D are block diagrams that illustrate examplemethods for tracking an object with a camera coupled to a gimbal mountedon a rotating platform.

FIG. 11 illustrates an example of a gimbal coupled to a pole mountapparatus.

FIG. 12 is a block diagram that illustrates an example method forstabilizing video recorded by a camera attached to mount platform via agimbal.

DETAILED DESCRIPTION

The Figures (FIGS.) and the following description relate to preferredembodiments by way of illustration only. It should be noted that fromthe following discussion, alternative embodiments of the structures andmethods disclosed herein will be readily recognized as viablealternatives that may be employed without departing from the principlesof what is claimed.

Reference will now be made in detail to several embodiments, examples ofwhich are illustrated in the accompanying figures. It is noted thatwherever practicable similar or like reference numbers may be used inthe figures and may indicate similar or like functionality. The figuresdepict embodiments of the disclosed system (or method) for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles described herein.

Configuration Overview

Disclosed by way of example embodiments is an electronic gimbalecosystem. The ecosystem may include a gimbal for use with a camera andmounting configurations. The gimbal may comprise a pivotable supportthat can mechanically stabilize an attached device along one or moreaxes. Particularly, in one embodiment, the gimbal comprises a 3-axisstabilization device for stabilizing a camera along pitch, roll, and yawaxes. The gimbal can include an inertial measurement unit that can sensethe orientation of the camera and a number of electronic motors (e.g.,three motors) which can manipulate the orientation of the camera. Thisorientation can correspond to the pitch, roll, and yaw of the camera.The gimbal can be removably coupled to a variety of mount platforms. Amount platform can comprise a structure for attaching to the gimbal or acamera. In an embodiment, the mount platform can comprise a stationaryor moveable device that may include a control interface for controllingits own movement, movement of the gimbal, or operation of the camera.For example, mount platforms may include an aerial vehicle, a handheldgrip, or a rotating mount. Moreover, the camera can be removably coupledto the gimbal and can be held in a removable camera frame.

Also disclosed is a system for allowing the mount platform to which thegimbal is mounted to control settings of the camera or to triggeractions on the camera, such as taking a picture, or initiating therecording of a video. The gimbal can also provide a connection betweenthe camera and the mount platform, such that the mount platform receivesimages and video content from the camera.

The mount platform can also control the movement of the gimbal, suchthat the gimbal performs differently on different mount platforms. Aplatform-specific behavior of the gimbal can allow the camera mounted tothe gimbal to capture images and video in a way best suited to the typeof mount platform.

In one embodiment, a mount may connect a gimbal to a mount platform. Themount may include a fixed mount floor that may be rigidly attachable tothe mount platform and a fixed mount ceiling that may be rigidlyattachable to the mount platform such that a gap may exist between a topsurface of the fixed mount floor and a bottom surface of the fixed mountceiling. A plurality of elastic connectors may protrude from the bottomsurface of the fixed mount ceiling. A floating base may have a topsurface mechanically coupled to the plurality of elastic connectors andmay hang below the fixed mount ceiling and adjacent to the fixed mountfloor. A gimbal connection housing may rigidly attach to the floatingbase. The gimbal connection housing may be removably connectable to thegimbal. A plurality of locking blocks may protrude from the floatingbase towards the fixed mount floor. A plurality of locking slots in thefixed mount floor may be structured to form a cavity reciprocal to theplurality of locking blocks. At an equilibrium position without a netcontact force applied to the floating base in a direction toward thefixed mount floor, a gap may exist between ends of the plurality oflocking blocks and corresponding ends of the plurality of locking slots.Furthermore, at a non-equilibrium position when a net contact force isapplied to the floating base in the direction towards the fixed mountfloor, the ends of the plurality of locking blocks may be flush with thecorresponding ends of the plurality of locking slots.

In another embodiment, a dampening base may couple a gimbal to a mountplatform. A floating base may include a gimbal connection housing thatmay be configured to enclose an end of the gimbal. The floating base maybe connectable to the mount platform by a plurality of elasticconnectors. A locking mechanism may be connected to the floating base.The locking mechanism in a locked position may be capable of rigidlycoupling the end of the gimbal to the gimbal connection housing. Thelocking mechanism in an unlocked position may be capable of allowing thegimbal to be removed from the gimbal connection housing. A plurality oflocking blocks may be connected to the floating base. In an absence of acontact force applied to the gimbal pushing the end of the gimbal intothe gimbal connection housing, each locking block of the plurality oflocking blocks may be held at a first position relative to acorresponding locking slot of a plurality of locking slots in the mountplatform. When the contact force is applied to the gimbal, each lockingblock of the plurality of locking blocks may be moved into a secondposition wherein each locking block of the plurality of locking blocksmay be flush with the corresponding locking slot of the plurality oflocking slots.

In another embodiment, an aerial vehicle system may include an aerialvehicle platform and a fixed mount portion that may be rigidly attachedto the aerial vehicle platform. A plurality of elastic connectors mayprotrude from a bottom surface of the fixed mount portion. A floatingbase may have a top surface mechanically coupled to the plurality ofelastic connectors. A plurality of locking blocks may protrude from thefloating base. A plurality of locking slots in a side surface of thefixed mount portion may be structured to form a cavity reciprocal to theplurality of locking block. A gimbal connection housing may be rigidlyattached to the floating base. A gimbal, may be removably attached tothe gimbal connection housing.

In another embodiment, a rotating mount apparatus may comprise a gimbalmount, a motor shaft, an electric motor, and a base. The gimbal mountmay include a mechanical coupling for removably coupling to a gimbal.The gimbal mount further may include an electronic connection that mayconnect to the gimbal. The motor shaft may be connected to the gimbalmount such that rotation of the motor shaft may cause the gimbal mountto rotate. The electric motor may be coupled to the motor shaft toprovide torque on the motor shaft. The base may be coupled to theelectric motor.

In another embodiment, an electronic gimbal may include a mountconnection that may have an electronic mount platform connectionconfigured to connect to a mount platform. A first motor may beconnected to the mount connection. The first motor may include a firstmotor shaft. Torque may be applied by the first motor to rotate thefirst motor shaft. A second motor may be connected to the first motorshaft. The second motor may include a second motor shaft. Torque may beapplied by the second motor to rotate the second motor shaft. A thirdmotor may be connected to the second motor shaft. The third motor mayinclude a third motor shaft. Torque may be applied by the third motor torotate the third motor shaft. An electronic camera connection may beconnected to the third motor shaft. The electronic camera connection maybe capable of connecting to a digital camera mounted on the electronicgimbal. An internal data bus may connect the electronic cameraconnection to the electronic mount platform connection.

In another embodiment, a stabilizing mounting system for a camera mayinclude a handheld grip and an electronic gimbal. The handheld grip mayinclude a shaft, a gimbal connection, and a control button. The gimbalconnection may be at an end of the shaft and may include a firstsecuring mechanism and a first electrical interface. The control buttonmay be on the shaft and when activated may cause a control signal to betransmitted via the gimbal connection. The electronic gimbal maycomprise a grip connection, a first motor, a second motor, a thirdmotor, a camera connection, and an internal data base. The gripconnection may include a second securing mechanism that may removablysecure to the first securing mechanism of the handheld grip and a secondelectrical interface that may communicatively couple to the firstelectrical interface of the handheld grip. The first motor may beconnected to the grip connection. The first motor may apply a firsttorque to a first motor shaft to cause the first motor shaft to rotateabout a first axis of rotation. The second motor may be connected to thefirst motor shaft. The second motor may apply a second torque to asecond motor shaft to rotate the second motor shaft about a second axisof rotation. The third motor may be connected to the second motor shaft.The third motor may apply a third torque to a third motor shaft torotate the third motor about a third axis of rotation. The cameraconnection may include a third securing mechanism that may removablysecure a camera to the third motor shaft of the electronic gimbal. Thecamera connection may furthermore comprise a third electrical interfacethat may communicatively couple the electronic gimbal to the camera. Theinternal data bus may communicatively connect the second electricalinterface to the third electrical interface. The internal data bus mayfurthermore transfer the control signal from the handheld grip to thecamera when the control button is activated to enable the control buttonon the handheld grip to control an action of the camera.

In another embodiment of the stabilizing mounting system, the handheldgrip may comprise a shaft and a gimbal connecting means for connectingto a gimbal. The gimbal connecting means may comprise a first securingmeans for mechanically securing to the gimbal and may comprise a firstelectrical interfacing means for electrically interfacing to the gimbal.The control means on the shaft may cause a control signal to betransmitted via the gimbal connecting means. The electronic gimbal maycomprise a grip connecting means for connecting to the handheld grip.The grip connecting means may include a second securing means forremovably securing to the first securing means of the handheld grip andmay include a second electrical interfacing means for communicativelyinterfacing to the first electrical interfacing means of the handheldgrip. A first rotating means may be connected to the grip connectingmeans. The first rotating means may apply a first torque to a firstmotor shaft that may cause the first motor shaft to rotate about a firstaxis of rotation. A second rotating means may be connected to the firstmotor shaft. The second rotating means may apply a second torque to asecond motor shaft to rotate the second motor shaft about a second axisof rotation. A third rotating means may be connected to the second motorshaft. The third rotating means may apply a third torque to a thirdmotor shaft to rotate the third motor about a third axis of rotation. Acamera connecting means may connect to a camera. The camera connectingmeans may include a third securing means that may removably secure thecamera to the third motor shaft. The camera connecting means may furthercomprise a third electrical interfacing means for communicativelycoupling the electronic gimbal to the camera. A data transfer means maycommunicatively connect the second electrical interfacing means to thethird electrical interfacing means. The data transfer means mayfurthermore transfer the control signal from the handheld grip to thecamera when the control means is activated which may enable the controlmeans on the handheld grip to control an action of the camera.

In another embodiment, a device may comprise a camera frame allowing acamera to be rigidly and removably coupled to the camera frame. A firstcontrol connection may electrically couple to the camera. The firstcontrol connection may be located on an inner side of the camera frame.A second control connection may electrically couple to a gimbal totransmit control signals from the gimbal to the camera through the firstcontrol connection. The second control connection may be located on anouter side of the camera frame. A first video connection mayelectrically couple to the camera. The first video connection may belocated on the inner side of the camera frame. A second video connectionmay electrically couple to the gimbal to transmit video data from thecamera to the gimbal through the first video connection. The secondvideo connection may be located on the outer side of the camera frame.

In another embodiment, a method, apparatus, or non-transitory computerreadable storage medium may control a camera to track an object. Acamera may be coupled to an electronic gimbal, which may be coupled to arotating platform. A first angular position of the object relative to anorientation of the camera may be detected. A desired motion state of thecamera for tracking the object may be determined and a motion state ofthe camera may be determined. A motor of the rotating platform, a motorof the gimbal, or both may be controlled depending on the desired motionstate of the camera and the motion state of the camera in a manner thatmay reduce a difference between the motion state of the camera and thedesired motion state of the camera.

In another embodiment, a method, apparatus, or non-transitory computerreadable storage medium may control electronic image stabilization in acamera attached to an electronic gimbal. While the camera may becapturing a video, a camera orientation error may be detected which mayrepresent a difference between a target orientation of the camera and adetected orientation of the camera. The camera orientation error may becompared to an error threshold. Responsive to the camera orientationerror exceeding the error threshold, a high response gimbal controlscheme may be applied to mechanically stabilize the camera and theelectronic image stabilization may be enabled to stabilize capture ofthe video. Responsive to the camera orientation not exceeding the errorthreshold, an available power budget may be compared to a power budgetthreshold. Responsive to the available power budget exceeding the powerbudget threshold, the high response gimbal control scheme may be appliedto mechanically stabilize the camera by reducing the camera orientationerror and the electronic image stabilization may be disabled ifpreviously enabled. Responsive to the available power budget notexceeding the power budget threshold, a low response gimbal controlscheme may be applied to mechanically stabilize the camera by reducingthe camera orientation error and the electronic image stabilization maybe enabled. The low response gimbal control scheme may use less averagepower than the high response gimbal control scheme. The high responsegimbal control scheme may decrease the camera orientation error morequickly than the low response gimbal control scheme.

Example System Configuration

Figure (FIG. 1 is a functional block diagram illustrating an examplesystem framework. In this example, the gimbal system 160 includes agimbal 100, a mount platform 110, a camera 120, a detachable frame 130,a camera control connection 140 and a camera output connection 141, anda gimbal control system 150. The gimbal 100 may include a sensor unit101 and a control logic unit 102. The mount platform 110 may include acamera controller 111, an image/video receiver 112, and a control logicunit 113. The camera 120 may couple to the detachable camera frame 130which is mounted on the gimbal 100 which is, in turn, coupled to themount platform 110. The coupling between the gimbal 100 and the mountplatform 110 may include a mechanical coupling and a communicationcoupling. The communication coupling may comprise an electricalconnection that enables data to be exchanged between the gimbal 100 andthe mount platform 110 such as, for example, control information,audio/visual information, or other data. The camera control connection140 and a camera output connection 141 may electrically connect thecamera 120 to the mount platform 110 for communication coupling. Thecamera control connection 140 and a camera output connection 141 may becomposed of interconnecting electronic connections and data busses inthe mount platform 110, gimbal 100, detachable camera frame 130 andcamera 120. The gimbal control system 150 may control the gimbal 100using a combination of a sensor unit 101 and a gimbal control logic unit102 in the gimbal 100 and a mount platform control logic unit 113 in themount platform 110.

The camera 120 can include a camera body, one or more a camera lenses,various indicators on the camera body (such as LEDs, displays, and thelike), various input mechanisms (such as buttons, switches, andtouch-screen mechanisms), and electronics (e.g., imaging electronics,power electronics, metadata sensors, etc.) internal to the camera bodyfor capturing images via the one or more lenses and/or performing otherfunctions. The camera 120 can capture images and videos at various framerates, resolutions, and compression rates. The camera 120 can beconnected to the detachable camera frame 130, which mechanicallyconnects to the camera 120 and physically connects to the gimbal 100.FIG. 1 depicts the detachable camera frame 130 enclosing the camera 120in accordance with some embodiments. In some embodiments, the detachablecamera frame 130 does not enclose the camera 120, but functions as amount to which the camera 120 couples. Examples of mounts include aframe, an open box, or a plate. Alternately, the detachable camera frame130 can be omitted and the camera 120 can be directly attached to acamera mount which is part of the gimbal 100.

The gimbal 100 is, in some example embodiments, an electronic three-axisgimbal which rotates a mounted object (e.g., a detachable camera frame130 connected to a camera 120) in space (e.g., pitch, roll, and yaw). Inaddition to providing part of an electronic connection between thecamera 120 and the mount platform 110, the gimbal may include a sensorunit 101 and a control logic unit 102, both of which are part of agimbal control system 150. In an embodiment, the gimbal control system150 detects the orientation of the gimbal 100 and camera 120, determinesa preferred orientation of the camera 120, and controls the motors ofthe gimbal in order to re-orient the camera 120 to the preferredorientation. The sensor unit 101 can include an inertial measurementunit (IMU) which measures rotation, orientation, and acceleration usingsensors, such as accelerometers, gyroscopes, and magnetometers. Thesensor unit 101 can also contain rotary encoders, which detect theangular position of the motors of the gimbal 100, and a magnetometer todetect a magnetic field, such as the earth's magnetic field. In someembodiments, the sensors of the sensor unit 101 are placed such as toprovide location diversity. For example, a set of accelerometers andgyroscopes can be located near the camera 120 (e.g., near the connectionto the detachable camera frame 130) and a set of accelerometers andgyroscopes can be placed at the opposite end of the gimbal (e.g., nearthe connection to the mount platform 110). The outputs of these two setsof sensors can be used by the IMU to calculate the orientation androtational acceleration of the camera, which can then be output to thegimbal control system 150. In some embodiments, the sensor unit 101 islocated on the mount platform 110. In some embodiments, the gimbalcontrol system 150 receives data from sensors (e.g., an IMU) on themount platform 110 and from the sensor unit 101 of the gimbal 100. Insome embodiment the sensor unit 101 does not include an IMU and insteadreceives position, acceleration, orientation, and/or angular velocityinformation from an IMU located on the camera 120.

The gimbal control logic unit 102, the sensor unit 101, and the mountplatform control logic unit 113 on the mount platform 110 constitute agimbal control system 150, in one embodiment. As discussed above, theIMU of the sensor unit 101 may produce an output indicative of theorientation, angular velocity, and acceleration of at least one point onthe gimbal 100. The gimbal control logic unit 102 may receive the outputof the sensor unit 101. In some embodiments, the mount platform controllogic unit 113 receives the output of the sensor unit 101 instead of, orin addition to the gimbal control logic unit 102. The combination of thegimbal control logic unit 102 and the mount platform control logic unit113 may implement a control algorithm which control the motors of thegimbal 100 to adjust the orientation of the mounted object to apreferred position. Thus, the gimbal control system 150 may have theeffect of detecting and correcting deviations from the preferredorientation for the mounted object.

The particular configuration of the two control portions of the gimbalcontrol system 150 may vary between embodiments. In some embodiments,the gimbal control logic unit 102 on the gimbal 100 implements theentire control algorithm and the mount platform control logic unit 113provides parameters which dictate how the control algorithm isimplemented. These parameters can be transmitted to the gimbal 100 whenthe gimbal 100 is originally connected to the mount platform 110. Theseparameters can include a range of allowable angles for each motor of thegimbal 100, the orientation, with respect to gravity, that each motorshould correspond to, a desired angle for at least one of the threespacial axes with which the mounted object should be oriented, andparameters to account for different mass distributions of differentcameras. In another embodiment, the mount platform control logic unit113 performs most of the calculations for the control algorithm and thegimbal control logic unit 102 includes proportional-integral-derivativecontrollers (PID controllers). The PID controllers' setpoints (i.e., thepoints of homeostasis which the PID controllers target) can becontrolled by the mount platform control logic unit 113. The setpointscan correspond to motor orientations or to the orientation of themounted object. In some embodiments, either the gimbal control logicunit 102 or the mount platform control logic unit 113 may be omitted,and the control algorithm may be implemented entirely by the othercontrol logic unit.

The mount platform 110 is shown electrically and mechanically connectedto the gimbal 100. The mount platform 110 may be, for example, an aerialvehicle, a handheld grip, a land vehicle, a rotating mount, a polemount, or a generic mount, each of which can itself be attached to avariety of other platforms. The gimbal 100 may be capable of removablycoupling to a variety of different mount platforms. The mount platform110 can include a camera controller 111, an image/video receiver 112,and the aforementioned control logic unit 113. The image/video receiver112 can receive content (e.g., images captured by the camera 120 orvideo currently being captured by the camera 120). The image/videoreceiver 112 can store the received content on a non-volatile memory inthe mount platform 110. The image/video receiver 112 can also transmitthe content to another device. In some embodiments, the mount platform110 transmits the video currently being captured to a remote controller,with which a user controls the movement of the mount platform 110, via awireless communication network.

The gimbal 100 can be electrically coupled the camera 120 and to themount platform 110 in such a way that the mount platform 110 (e.g., aremote controlled aerial vehicle or a hand grip) can generate commandsvia a camera controller 111 and send the commands to the camera 120.Commands can include a command to toggle the power the camera 120, acommand to begin recording video, a command to stop recording video, acommand to take a picture, a command to take a burst of pictures, acommand to set the frame rate at which a video is recording, or acommand to set the picture or video resolution. Another command that canbe sent from the mount platform 110 through the gimbal 100 to the camera120 can be a command to include a metadata tag in a recorded video or ina set of pictures. In this exemplary configuration, the metadata tagcontains information such as a geographical location or a time. Forexample, a mount platform 110 can send a command through the gimbal 100to record a metadata tag while the camera 120 is recording a video. Whenthe recorded video is later played, certain media players may beconfigured to display an icon or some other indicator in associationwith the time at which the command to record the metadata tag was sent.For example, a media player might display a visual queue, such as anicon, along a video timeline, wherein the position of the visual queuealong the timeline is indicative of the time. The metadata tag can alsoinstruct the camera 120 to record a location, which can be obtained viaa GPS receiver (Global Positioning Satellite receiver) located on themount platform 110, the camera 120, or elsewhere, in a recorded video.Upon playback of the video, the metadata can be used to map ageographical location to the time in a video at which the metadata tagwas added to the recording.

Signals, such as a command originating from the camera controller 111 orvideo content captured by a camera 120 can be transmitted throughelectronic connections which run through the gimbal 100. In someembodiments, telemetric data from a telemetric subsystem of the mountplatform 110 can be sent to the camera 120 to associate with image/videocaptured and stored on the camera 120. A camera control connection 140can connect the camera controller 111 module to the camera 120 and acamera output connection 141 can allow the camera 120 to transmit videocontent or pictures to the image/video receiver 112. The electronicconnections can also provide power to the camera 120, from a batterylocated on the mount platform 110. The battery of the mount platform 110can also power the gimbal 100. In an alternate embodiment, the gimbal100 contains a battery, which can provide power to the camera 120. Theelectrical connections between the camera 120 and the gimbal 110 can runthrough the gimbal 100 and the detachable camera frame 130. Theelectrical connections between the camera 120 and the mount platform 110can constitute a daisy chain or multidrop topology in which the gimbal100 and detachable camera frame 130 act as buses. The electricalconnections can implement various protocols such as HDMI(High-Definition Multimedia Interface), USB (Universal Serial Bus), orEthernet protocols to transmit data. In one embodiment, the cameraoutput connection 141 transmits video data from the camera 120 via theHDMI protocol connection and the camera control connection 140 is a USBconnection. In some embodiments, the electrical connection between thecamera 120 and the mount platform 110 is internal to the gimbal 100. Forexample, in one embodiment, a data bus is substantially enclosed in thegimbal 100 and may be exposed at an interface at either end using, forexample, a male or female interface connector.

In one embodiment, an electrical signal or mechanical mechanism mayenable the gimbal to detect what type of mounting platform 110 it isconnected to so that it can configure itself accordingly. For example, acontrol signal may be sent form the mounting platform 110 to the gimbal100 identifying the platform type. Alternatively, the gimbal 100 maydetect what type of mounting platform 110 it is connected to duringusage based on motion or other sensor data. For example, the gimbal 100can detect whether its motion is more consistent with an aerial vehicleor handheld grip.

Example Aerial Vehicle Configuration

FIG. 2 illustrates an embodiment in which the mount platform 110 is anaerial vehicle 200. More specifically, the mount platform 110 in thisexample is a quadcopter (i.e., a helicopter with four rotors). Theaerial vehicle 200 in this example includes housing 230 which encloses apayload (e.g., electronics, storage media, and/or camera), four arms235, four rotors 240, and four propellers 245. Each arm 235 maymechanically couple with a rotor 240, which in turn couples with apropeller 245 to create a rotary assembly. When the rotary assembly isoperational, all the propellers 245 may rotate at appropriate speeds toallow the aerial vehicle 200 lift (take off), land, hover, and move(forward, backward) in flight. Modulation of the power supplied to eachof the rotors 240 can control the trajectory and torque on the aerialvehicle 200.

A gimbal 100 is shown coupled to the aerial vehicle 200. A camera 120 isshown enclosed in a removable camera frame 130 which is attached thegimbal 100. The gimbal 100 may be mechanically and electrically coupledto the housing 230 of the aerial vehicle 200 through a removablecoupling mechanism that mates with a reciprocal mechanism on the aerialvehicle 200 having mechanical and communicative capabilities. The gimbal100 can be removed from the aerial vehicle 200. The gimbal 100 can alsobe removably attached to a variety of other mount platforms, such as ahandheld grip, a ground vehicle, and a generic mount, which can itselfbe attached to a variety of platforms. In some embodiments, the gimbal100 can be attached or removed from a mount platform 110 without the useof tools.

In an embodiment, the aerial vehicle 200 includes a battery that can beused to provide power to the camera 120, the gimbal 100, or both.

Example Gimbal

FIGS. 3A and 3B, illustrate an exemplary embodiment of the gimbal 100attached to a removable camera frame 130, which itself is attached to acamera 120. The example gimbal 100 includes a base arm 310, a middle arm315, a mount arm 320, a first motor 301, a second motor 302, and a thirdmotor 303. Each of the motors 301, 302, 303 can have an associatedrotary encoder, which will detect the rotation of the axel of the motor.Each rotary encoder can be part of the sensor unit 101. The base arm 310can be configured to include a mechanical attachment portion 350 at afirst end that allows the gimbal 100 to securely attach a reciprocalcomponent on another mount platform (e.g., an aerial vehicle 200, aground vehicle, or a handheld grip), and also be removable. The base arm310 also includes the first motor 301. The base arm 310 may mechanicallycouple to the middle arm 315. A first end of the middle arm 315 maymechanically couple to the first motor 301. A second end of the middlearm 315 may mechanically couple to the second motor 302. A first end ofthe mount arm 320 may mechanically couple to the second motor 302. Thesecond end of the mount arm 320 may mechanically couple to the thirdmotor 303 which may mechanically couple to the camera frame 130. Withinthe camera frame 130, the camera 120 may be removably secured.

The gimbal 100 may be configured to allow for rotation of a mountedobject in space. In the embodiment depicted in FIG. 3A and FIG. 3B, themounted object is a camera 120 to which the gimbal 100 is mechanicallycoupled. The gimbal 100 may allow for the camera 120 to maintain aparticular orientation in space so that it remains relatively steady asthe platform to which it is attached moves (e.g., as an aerial vehicle200 tilts or turns during flight). The gimbal 100 may have three motors,each of which rotates the mounted object (e.g., the camera 120) about aspecific axis of rotation. Herein, for ease of discussion, the motorsare numbered by their proximity to the mount platform 110 (i.e., thefirst motor 301, the second motor 302, and the third motor 303).

The gimbal control system 150 may control the three motors 301, 302, and303. After detecting the current orientation of the mounted object, viathe sensor unit 101, the gimbal control system 150 can determine apreferred orientation along each of the three axes of rotation (i.e.,yaw, pitch, and roll). The preferred orientation may be used by thegimbal control system 150 to compute a rotation for each motor in orderto move the camera 120 to the preferred orientation or keep the camera120 in the preferred orientation. In one embodiment, the gimbal controlsystem 150 has a preferred orientation that is configured by the user.The user can input the preferred orientation of the camera 120 with aremote controller. For example, the user can input the preferredorientation with a remote controller for a mounting platform 110, whichsends the preferred orientation for the camera 120 to the mountingplatform 110 (e.g., aerial vehicle 200) through a wireless network,which then provides the preferred orientation to the gimbal controlsystem 150. In some example embodiments, an orientation can be definedrelative to the ground, so that the yaw, pitch, and roll of the cameraremain constant relative to the ground. In some embodiments, certainaxes of rotation can be unfixed. That is, an unfixed axis of rotationmay not be corrected by the gimbal control system 150, but rather mayremain constant relative to the aerial vehicle 200. For example, the yawof the camera 120 can be unfixed, while the roll and the pitch arefixed. In this case, if the yaw of the aerial vehicle 200 changes theyaw of the camera 120 will likewise change, but the roll and the pitchof the camera 120 will remain constant despite roll and pitch rotationsof the aerial vehicle 200.

In some example embodiments, bounds of rotation can be defined whichlimit the rotation along certain axes relative to the connection betweenthe gimbal 110 and the mount platform 110. For example, if α_(max) andα_(min) are the relative maximum and minimum values for the yaw of thecamera 120 relative to the mount platform 110, then if the aerialvehicle 200 is oriented at a yaw of α_(av) degrees, the preferred yaw ofthe camera α_(c) may be chosen by the gimbal control system 150 so thatthe angle α_(c) is between the angles (α_(min)+α_(av)) and(α_(max)+α_(av)). Similar maximum and minimum values can be defined forthe pitch and roll. The maximum and minimum for each of the relativeangles can be defined such that the viewing angle of the camera 120 isnot obstructed by the gimbal 100 and/or the mount platform 110 at anyangle within the valid bounds. In some embodiments, the preferredorientation of the camera 120 is defined using one or more trackingalgorithms, which will be further discussed herein.

The axis to which each motor corresponds can depend on the mountplatform 110 to which the gimbal 100 is attached. For example, whenattached to the aerial vehicle 200, the first motor 301 can rotate themounted object about the roll axis, the second motor 302 can rotate themounted object about the yaw axis and the third motor 303 can rotate themounted object about the pitch axis. However, when the same gimbal 100is attached to a handheld grip, the motors correspond to different axes:the first motor 301 can correspond to yaw axis, and the second motor 302can corresponds to roll axis, while the third motor 303 can stillcorresponds to pitch axis.

In some embodiments, each of the three motors 301, 302, 303 isassociated with an orthogonal axis of rotation. However, in otherembodiments, such as the embodiment depicted in FIG. 3A and FIG. 3B themotors 301, 302, 303 of the gimbal 100 are not orthogonal. A gimbal 100in which the motors are not orthogonal may have at least one motor thatrotates the mounted object about an axis which is not orthogonal to theaxis of rotation of the other motors. In a gimbal 100 in which themotors are not orthogonal, operation of one motor of the gimbal 100 cancause the angle of the camera 120 to shift on the axis of another motor.In the example embodiment shown in FIG. 3A and FIG. 3B, the first motor301 and the third motor 303 have axes of rotation that are orthogonal toeach other, and the second motor 302 and the third motor 303 areorthogonal, but the first motor 301 and second motor 302 are notorthogonal. Because of this configuration, when the gimbal 100 iscoupled to the aerial vehicle 200 and the aerial vehicle 200 is level,operation of the first motor 301 may adjust only the roll of the camera120 and the third motor 303 may adjust only the pitch of the camera 120.The second motor 302 may adjust the yaw primarily, but also may adjustthe pitch and roll of the camera 120. Suppose for the purpose ofexample, the gimbal 100 is attached to the aerial vehicle 200 and thecamera 120 is initially oriented at a pitch, yaw, and roll of 0° andthat the axis of the second motor 302 is orthogonal to the axis of thethird motor 303 and forms an angle of θ degrees with the vertical axis,as depicted in FIG. 3B. In FIG. 3B, the angle θ is measured clockwise,and is about 16°. A rotation of ϕ degrees (where −180°≤ϕ≤180° by thesecond motor 302 may also change the pitch, p, of the camera 120 top=(|ϕ|*θ)/90° where a pitch greater than 0 corresponds to the camerabeing oriented beneath the horizontal plane (i.e., facing down). Therotation of the second motor 302 by ϕ degrees may also change the roll,r, of the camera 120 to r=θ*(1−|ϕ−90°|/90°) in the case where−90°≤ϕ≤180° and the roll will change to r=−(θ*ϕ)/90°−θ in the case where−180°<ϕ<−90°. The change in the yaw, y, of the camera 120 may beequivalent to the change in angle of the second motor 120 (i.e., y=ϕ).

A non-orthogonal motor configuration of the gimbal 100 can allow for alarger range of unobstructed viewing angles for the camera 120. Forexample, in the embodiment shown in FIG. 3A and FIG. 3B, the pitch ofthe camera 120 relative to the connection of the gimbal 100 to the mountplatform 110 (e.g., aerial vehicle 200) can be about 16° higher withoutthe camera's frame being obstructed (i.e., without the motor appearingin the image captured by the camera) than it could with an orthogonalmotor configuration. In some embodiments, the second motor 302 may notbe identical to the other two motors 301, 303. The second motor 302 canbe capable of producing a higher torque than the other two motors 301,303. In another embodiment, a different one of the motors 301, 302, 303may be capable of producing a higher torque than the other two motors.In another embodiment, all three motors 301, 302, 303 may be capable ofproducing different amounts of torque. In yet another embodiment, allthree motors 301, 302, 303 may be capable of producing substantiallysimilar torques.

A larger value of θ (the angle between the second motor 302 and the axisorthogonal to the rotational axes of the other two motors) in anon-orthogonal motor configuration can provide a larger range of viewingangles for the mounted camera 120, but a larger θ will result in ahigher maximum torque than a comparable orthogonal motor configuration.Thus, embodiments in which the motors are not orthogonal can implement avalue of θ in which the two design considerations of a large viewingangle for the camera 120 and the torque from the motors are optimized.Consequently, the choice of θ will depend on many factors, such as thetargeted price point of the gimbal 100, the type of cameras supported,the desired use cases of the gimbal, the available motor technology,among other things. It is noted that by way of example, θ can be between0°≤θ≤30°. In another embodiment, θ can be between 5°≤θ≤30°. Other rangesare also possible.

The gimbal 100 can support a plurality of different cameras withdifferent mass distributions. Each camera can have a correspondingdetachable camera frame (e.g., camera 120 corresponds to the detachablecamera frame 130), which secures the camera. A detachable camera frame130 may have an electrical connector, or a multiplicity of electricalconnectors, which couple to the gimbal 100 and an electrical connector,or a multiplicity of electrical connectors, which couple to the camera120. Thus, the detachable camera frame 130 may include a bus for sendingsignals from the camera to the gimbal 100, which can, in some cases, berouted to the mount platform 110. In some embodiments, each detachablecamera frame has the same types of electrical connectors for coupling tothe gimbal 100, but the type of electrical connector that connects tothe camera is specific to the type of camera. In another embodiment, thedetachable camera frame 130 provides no electronic connection betweenthe camera 120 and the gimbal 100, and the camera 120 and gimbal 100 aredirectly electrical connected (e.g., via a cable). In some embodiments,the gimbal 100 does not contain a bus and the camera 120 and the mountplatform 110 communicate via a wireless connection (e.g., Bluetooth orWiFi).

In some example embodiments, the gimbal 100 may have a mount connector304 (shown in FIG. 3B, but not in FIG. 3A) which allows the gimbal 100to electronically couple to the mount platform 110 (e.g., the aerialvehicle 200). The mount connector 304 can include a power connectionwhich provides power to the gimbal 100 and the camera 120. The mountconnector 304 can also allow communication between the sensor unit 101and the gimbal control logic unit 102 on the gimbal 100 and the mountplatform control logic unit 113 on the mount platform 110. In someembodiments, the mount connector 304 electrically connects to the camera120 via busses (e.g., a camera control connection 140 and a cameraoutput connection 141) which allow communication between the mountplatform 110 and the camera 120.

The gimbal 100 also can couple mechanically to a mount platform 110 viaa mechanical attachment portion 350. In an embodiment, the gimbal 100 isa modular device that can be quickly and easily connected anddisconnected from a mounting platform 350 (e.g., aerial vehicle 200,handheld grip, rotating mount, etc.). For example, in one embodiment,mechanical attachment portion 350 comprises a quick-release mechanism orother mechanism that does not require tools. The mechanical attachmentportion 350 can be part of the base arm 310. The mechanical attachmentportion 350 can include a mechanical locking mechanism to securelyattach a reciprocal component on a mount platform 110 (e.g., an aerialvehicle 200, a ground vehicle, an underwater vehicle, or a handheldgrip). The example mechanical locking mechanism shown in FIGS. 3A and 3Bincludes a groove with a channel in which a key (e.g., a tapered pin orblock) on a reciprocal component on a mount platform 110 can fit. Thegimbal 100 can be locked with the mount platform 110 in a first positionand unlocked in a second position, allowing for detachment of the gimbal100 from the mount platform 110. The mechanical attachment portion 350may mechanically connect to a reciprocal component on a mount platform110 in which the mechanical attachment portion 350 may be configured asa female portion of a sleeve coupling and where the mount platform 110may be configured as a male portion of a sleeve coupling. Alternatively,the mechanical attachment portion 350 may be configured as a maleportion of a sleeve coupling and the mount platform may be configured afemale portion of a sleeve coupling. The mechanical coupling between themount platform 110 and the gimbal 100 can be held together by africtional force. The mechanical coupling between the mount platform 110and the gimbal 100 can also be held together by a clamping mechanism onthe mount platform 110.

If the gimbal 100 supports multiple different cameras of differing massdistributions, the differences in mass and moments of inertia betweencameras might cause the gimbal 100 to perform sub-optimally. A varietyof techniques are suggested herein for allowing a single gimbal 100 tobe used with cameras of different mass distributions. The detachablecamera frame 130 can hold the camera 120 in such a way that thedetachable frame 130 and camera 120 act as a single rigid body. In someexample embodiments, each camera which can be coupled to the gimbal 100has a corresponding detachable frame, and each pair of camera and framehave masses and moments of inertia which are approximately the same. Forexample, if m_(ca) and m_(fa) are the masses of a first camera and itscorresponding detachable frame, respectively, and if m_(cb) and m_(fb)are the masses of a second camera and its corresponding detachableframe, then, m_(ca)+m_(fa)≈m_(cb)+m_(fb). Also, I_(ca) and I_(fa) arethe matrices representing the moments of inertia for the axes aroundabout which the first camera rotates for the first camera and thecorresponding detachable frame, respectively. In addition, I_(cb) andI_(fb) are the corresponding matrices for the second camera and thecorresponding detachable frame, respectively. Thereafter,I_(ca)+I_(fa)≈I_(cb)+I_(fb), where “+” denotes the matrix additionoperator.) Since the mounted object which is being rotated by the gimbalis the rigid body of the camera and detachable camera frame pair, themass profile of the mounted object does not vary although the massprofile of the camera itself does. Thus, by employing detachable cameraframes e.g., 130, with specific mass profiles a single gimbal 100 cancouple to a multiplicity of cameras with different mass profiles.

In alternate embodiments, the mass profile of the camera 120 anddetachable frame 130 pair is different for each different type ofcamera, but control parameters used in the control algorithms,implemented by the gimbal control system 150, which control the motors,are changed to compensate for the different mass profiles of each paircamera and detachable camera frame. These control parameters can specifythe acceleration of a motor, a maximum or minimum for the velocity of amotor, a torque exerted by a motor, a current draw of a motor, and avoltage of a motor. In one embodiment, the camera 120 and/or the cameraframe 130 is communicatively coupled to either the gimbal 100 or themount platform 110, and upon connection of a camera 120 to the gimbal100 information is sent from the camera 120 to the gimbal control system150 which initiates the update of control parameters used to control themotors of the gimbal 100. The information can be the control parametersused by the gimbal control system 150, information about the massprofile (e.g., mass or moment of inertia) of the camera 120 and/ordetachable camera mount 130, or an identifier for the camera 120 or thecamera mount 130. If the information sent to the gimbal control system150 is a mass profile, then the gimbal control system 150 can calculatecontrol parameters from the mass profile. If the information is anidentifier for the camera 120 or the detachable camera frame 130, thegimbal control system 150 can access a non-volatile memory which storessets of control parameters mapped to identifiers in order to obtain thecorrect set of control parameters for a given identifier.

In some embodiments, the gimbal 100 may be capable of performing anauto-calibration sequence. This auto-calibration sequence may beperformed in response to a new camera 120 being connected to the gimbal100, in response to an unrecognized camera 120 being attached to thegimbal 100, in response to a new mount platform 110 being connected tothe gimbal, or in response to an input from a user. Auto-calibration mayinvolve moving the gimbal 100 to a number of set orientations. The speedat which the gimbal re-orients the camera 120 can be measured andcompared to an expected speed. The torque exerted by the motor, thecurrent draw of the motor, the voltage used to motor can be adjusted sothat the movement of the gimbal 100 is desirable.

In some embodiments, the movement characteristics of the gimbal 100 maybe adjusted according the type of mount platform 110 that the gimbal 100is connected to. For example, each type of mount platform 110 canspecify the maximum rotation speed of the gimbal 100, the maximum torqueapplied by the motors 301, 302, 303, or the weight given to theproportional, integral, and derivative feedback components used in a PIDcontroller used to control a motor 301, 302, or 303. In someembodiments, the motor power used for motion dampening is determinedbased on the type of connected mount platform 110. Furthermore, thegimbal 100 may operate within different angle ranges along each of theroll, pitch, and yaw dimensions depending on the mount platform 110. Forexample, the possible angles of rotation may include a wider range whenthe gimbal 100 is mounted to a handheld grip than when it is mounted toan aerial vehicle.

Furthermore, as a safety and self-protection parameter, in oneembodiment a motor power timeout may be triggered when excessiveresistance is detected on any motor axis for a given period of time.Furthermore, for power savings, the gimbal 100 may cut power to themotors when it detects a lack of movement indicating that it is not inuse. Power may be re-applied automatically when the gimbal 100 detectsthat it is in use again. Additionally, in one embodiment, the gimbal 100can only be powered on when it detects that is attached to both acompatible camera 120 and a compatible mounting platform 110 and whenthe mounting platform 110 can provide sufficient power to both devices.

In one embodiment, the gimbal control system 150 may obtain periodicfirmware updates. In one embodiment, the gimbal control system 150 mayreceive a firmware update via an attached handheld grip. For example,the handheld grip may receive the update via a connection (e.g., USB) toa computing device and the update may be flashed to the gimbal controlsystem 150 via the handheld grip. In another embodiment, the gimbalcontrol system 150 may be updated via a connected camera 120. In thiscase, the camera 120 may receive an update via a connected mobileapplication on a mobile device and subsequently transfer the update tothe gimbal control system 150. In yet another embodiment, when thegimbal 100 is being used with an aerial vehicle, an update may bereceived on a remote control operating the aerial vehicle. The remotecontrol alerts the user that an update is available and then wirelesslytransmits the update to the aerial vehicle, which in turn sends theupdate to the gimbal 100. In other embodiments, firmware updates may bereceived via other mounting platforms 120 or via other wired or wirelessconnections.

In an embodiment, the gimbal 100 is constructed of a highly durable(e.g., to withstand impact) and wear-resistant material for surfacefinishing. Furthermore, the gimbal 100 may be constructed of materialsrigid enough to limit sensor errors. Furthermore, the gimbal may besubstantially waterproof and flameproof In one embodiment, the gimbal100 has dimensions in the range of approximately 80-100 mm in width,70-90 mm in depth, and 80-100 mm in height.

Example Camera Architecture

FIG. 4 illustrates a block diagram of an example camera architecture.The example camera architecture 405 corresponds to an architecture forthe camera, e.g., 120. In one embodiment, the camera 120 is capable ofcapturing spherical or substantially spherical content. As used herein,spherical content may include still images or video having spherical orsubstantially spherical field of view. For example, in one embodiment,the camera 120 captures video having a 360° field of view in thehorizontal plane and a 180° field of view in the vertical plane.Alternatively, the camera 120 may capture substantially spherical imagesor video having less than 360° in the horizontal direction and less than180° in the vertical direction (e.g., within 10% of the field of viewassociated with fully spherical content). In other embodiments, thecamera 120 may capture images or video having a non-spherical wide anglefield of view.

As described in greater detail below, the camera 120 can include sensors440 to capture metadata associated with video data, such as timing data,motion data, speed data, acceleration data, altitude data, GPS data, andthe like. In a an example embodiment, location and/or time centricmetadata (geographic location, time, speed, etc.) can be incorporatedinto a media or image file together with the captured content in orderto track over time the location of the camera 120 or the subject beingrecorded. This metadata may be captured by the camera 120 itself or byanother device (e.g., a mobile phone, the aerial vehicle 200, or a datatracker worn by a subject such as a smart watch or fitness trackerequipped with tracking software or a dedicated radio frequency tracker)proximate to the camera 120. In one embodiment, the metadata may beincorporated with the content stream by the camera 120 as the sphericalcontent is being captured. In another embodiment, a metadata fileseparate from the video or image file may be captured (by the samecapture device or a different capture device) and the two separate filescan be combined or otherwise processed together in post-processing. Itis noted that these sensors 440 can be in addition to sensors in atelemetric subsystem of the aerial vehicle 200. In embodiments in whichthe camera 120 is integrated with the aerial vehicle 200, the cameraneed not have separate individual sensors, but rather could rely uponthe sensors integrated with the aerial vehicle 200 or another externaldevice.

In the embodiment illustrated in FIG. 4, the camera 120 may comprise acamera core 410 comprising a lens 412, an image sensor 414, and an imageprocessor 416. The camera 120 additionally may include a systemcontroller 420 (e.g., a microcontroller or microprocessor) that controlsthe operation and functionality of the camera 120 and system memory 430configured to store executable computer instructions that, when executedby the system controller 420 and/or the image processors 416, performthe camera functionalities described herein. In some embodiments, acamera 120 may include multiple camera cores 410 to capture fields ofview in different directions which may then be stitched together to forma cohesive image. For example, in an embodiment of a spherical camerasystem, the camera 120 may include two camera cores 410 each having ahemispherical or hyper hemispherical lens that each capture ahemispherical or hyper-hemispherical field of view which are stitchedtogether in post-processing to form a spherical image.

The lens 412 can be, for example, a wide angle lens, hemispherical, orhyper hemispherical lens that focuses light entering the lens to theimage sensor 414 which captures images and/or video frames. The imagesensor 414 may capture high-definition images having a resolution of,for example, 720p, 1080p, 4 k, or higher. In one embodiment, sphericalvideo is captured in a resolution of 5760 pixels by 2880 pixels with a360° horizontal field of view and a 180° vertical field of view. Forvideo, the image sensor 414 may capture video at frame rates of, forexample, 30 frames per second, 60 frames per second, or higher.

The image processor 416 performs one or more image processing functionsof the captured images or video. For example, the image processor 416may perform a Bayer transformation, demosaicing, noise reduction, imagesharpening, image stabilization, rolling shutter artifact reduction,color space conversion, compression, or other in-camera processingfunctions. The image processor 416 may be configured to performreal-time stitching of images, for example, when images are capturedfrom two or more cameras configured to capture images. Such exampleconfigurations may include, for example, an activity camera (which mayinclude a spherical image capture camera) with image sensors, each witha substantially different field of view (FOV), but where there may besome overlap where the images can be stitched together. Processed imagesand video may be temporarily or persistently stored to system memory 430and/or to a non-volatile storage, which may be in the form of internalstorage or an external memory card.

An input/output (I/O) interface 460 may transmit and receive data fromvarious external devices. For example, the I/O interface 460 mayfacilitate the receiving or transmitting video or audio informationthrough an I/O port. Examples of I/O ports or interfaces include USBports, HDMI ports, Ethernet ports, audio ports, and the like.Furthermore, embodiments of the I/O interface 460 may include wirelessports that can accommodate wireless connections. Examples of wirelessports include Bluetooth, Wireless USB, Near Field Communication (NFC),cellular (mobile) communication protocols, short range Wifi, etc., andthe like. The I/O interface 460 may also include an interface tosynchronize the camera 120 with other cameras or with other externaldevices, such as a remote control, a second camera, a smartphone, aclient device, or a video server.

A control/display subsystem 470 includes various control and displaycomponents associated with operation of the camera 120 including, forexample, LED lights, a display, buttons, microphones, speakers, and thelike. The audio subsystem 450 includes, for example, one or moremicrophones and one or more audio processors to capture and processaudio data correlated with video capture. In one embodiment, the audiosubsystem 450 includes a microphone array having two or more microphonesarranged to obtain directional audio signals.

Sensors 440 may capture various metadata concurrently with, orseparately from, video capture. For example, the sensors 440 may capturetime-stamped location information based on a global positioning system(GPS) sensor, and/or an altimeter. Other sensors 440 may be used todetect and capture orientation of the camera 120 including, for example,an orientation sensor, an accelerometer, a gyroscope, or a magnetometer.Sensor data captured from the various sensors 440 may be processed togenerate other types of metadata. For example, sensor data from theaccelerometer may be used to generate motion metadata, comprisingvelocity and/or acceleration vectors representative of motion of thecamera 120. Furthermore, sensor data from the aerial vehicle 200 and/orthe gimbal 100 may be used to generate orientation metadata describingthe orientation of the camera 120. Sensor data from a GPS sensor canprovide GPS coordinates identifying the location of the camera 120, andthe altimeter can measure the altitude of the camera 120.

In one example embodiment, the sensors 440 may be rigidly coupled to thecamera 120 such that any motion, orientation or change in locationexperienced by the camera 120 is also experienced by the sensors 440.The sensors 440 furthermore may associate one or more time stampsrepresenting when the data was captured by each sensor. In oneembodiment, the sensors 440 automatically begin collecting sensormetadata when the camera 120 begins recording a video.

In an embodiment, the camera 120 may be controlled by the mount platform110 or remotely, for example, through a remote controller, or throughother devices in wireless communication with the camera 120, eitherdirectly or through the mount platform 110. For example, the camera 120may be connected to an aerial vehicle 200, and control functions of thecamera 120 can be manipulated before, during or after flight (e.g., atlanding) by the aerial vehicle 200 or by a remote device wirelesslycommunicating with the camera 120 or aerial vehicle 200. For example,during flight the camera 120 can be configured to switch from shootingimages at 60 frames per second from 30 frames per second (fps). Theaerial vehicle 200 may follow a skier down a slope and start capturingimages through the camera 120 at 30 fps. As the skier accelerates, e.g.,for a jump, the camera 120 automatically switches to capturing images at60 fps. If the skier is in the distance, e.g., 20 meters, the camera 120may capture images at 30 fps, but as the aerial vehicle 200 drawscloser, e.g., within 5 meters, the camera 120 can automatically switchto capturing images at 60 fps.

The camera 120 can be partially enclosed or mounted to a detachablecamera frame 130, such as the detachable camera frame 500 depicted inFIG. 5. The detachable camera frame 500 may be structured to physicallycouple the camera 120 to the detachable camera frame 500 and also tophysically couple the detachable camera frame 500 to the gimbal 100,which is in turn coupled to a mount platform 110 such as an aerialvehicle 200, handheld grip, or rotating mount. In an embodiment, thecamera 120 can be easily inserted and removed from the detachable cameraframe 500 without the use of tools.

The detachable camera frame 500 also may include interfaces tofacilitate communications between the camera 110 and the mount platform110 via the gimbal 100. For example, the detachable camera frame 500includes a micro USB connector 510 and a HDMI connector 520 that cancouple with the corresponding camera (not shown). The USB connector 510,which can provide power to the camera 120 and can allow the mountplatform 110 (e.g., aerial vehicle 200) to send executable instructionsto the camera 120, such as a command to change the camera mode (e.g.,video, single photo, burst photo, time lapse photo, etc.) frame rate ofa video, or trigger the shutter to take a picture or start/stoprecording video. Furthermore, a command may be sent to insert ahighlight tag in the video during capture. The USB connector 510 mayalso charge or provide power to the camera 120 and remotely control thepower on/off state of the camera 120. Additionally, metadata from themount platform 110 (e.g., flight metadata from an aerial vehicle) may besent to the camera 110 via the USB connector 510 or other connector onthe detachable camera frame 500 to enable the metadata to be storedtogether with the captured video.

The HDMI connector 520 depicted may allow the camera 120 to transmitcaptured video, audio, and images to the mount platform 110. Thedetachable camera frame 500 can include any set of connectors andutilize any communication protocols to transmit data to and from themount platform 110. The detachable camera frame 500 can include a set ofconnectors (not shown) which connect to the gimbal 100, so that thegimbal 100 can act as a bus for transmitting data or power between themount platform 110 and the camera 120, and vice versa. The detachablecamera frame 500 may include a latch 530 for locking the camera into thedetachable camera frame 500. Detaching the latch 530 may allow thecamera to be removed or installed into the detachable camera frame 500.Locking the latch 530 may rigidly couple the camera to the detachablecamera frame 500.

The detachable camera frame 500 may be structured so that it does notobstruct the user's view of a rear display of the camera 120. Thus, forexample, in one embodiment, the detachable camera frame 500 may bestructured to secure around a perimeter of the camera 120 so as to notocclude the display of the camera 120.

In an embodiment, the detachable camera frame 500 may be constructed ofa highly durable a wear-resistant material for surface finishing.

Handheld Grip

FIG. 6 illustrates an example embodiment of a mount platform 110 thatcan removably couple with the gimbal 100. In this example, the mountplatform 110 may be a handheld grip 600 that electronically andmechanically couples with the gimbal 100. The handheld grip 600 includesa plurality of buttons 605, 610, 615, 620, 625 which can be used by auser to control the camera 120 and/or the gimbal 100. The handheld grip600 contains a battery from which it can provide power to the gimbal 100and may also be used to power and/or charge the camera 120, the gimbal100, or both in addition to operating any electronic functions on thehandheld grip 600 itself. For example, when all components are turned onand the handheld grip 600 is connected to a power supply, the grip 600may provide pass-through power to operate the gimbal 100 and the camera120 (via the gimbal 100). In one embodiment, the power may be sufficientto operate the camera 120 even if the camera battery is depleted or ifno camera battery is inserted. When the components are off, theconnected power supply charges both the battery in the handheld grip 600and the battery of the camera 120 via a power path through the gimbal100. In one embodiment, the camera 120 may be charged first until itreaches a threshold charge level, and the handheld grip may be chargedsecond. In one embodiment, the battery comprises a low discharge batterythat can at least at least several hours (e.g., at least 2-4 hours). Indifferent embodiments, the battery may be removable or integrated withthe grip 600.

In one embodiment, when the handheld grip 600 no longer has sufficientpower to supply to both the camera 120 and the gimbal 100, the handheldgrip 600 can send a control signal to the camera 120 to control thecamera 120 to save what it has recorded and then shut down the camera120, the gimbal 100, and the handheld grip 600.

The handheld grip 600 can be communicatively coupled to the camera 120via a connection provided by the gimbal 100. The camera 120 can providecaptured video content and images to the handheld grip 600. In oneembodiment, the handheld grip can store the provided video content andimages in storage media, such as a flash storage, which can be removablycoupled to the handheld grip 600 (e.g., a secure digital memory card (SDcard) or a micro SD card) or integrated into the handheld grip 600itself. In an alternate embodiment, the handheld grip 600 has a portwhich can be used to connect to another device, such as a personalcomputer. This port can allow the connected device to request andreceive video content and images from the camera 120. Thus, theconnected device, would receive content from the camera 120 via aconnection running through the detachable camera frame 130, the gimbal100, and the handheld grip 600. In some embodiments, the port on thehandheld grip 600 provides a USB connection. The USB connection or otherport may be used to supply power to the battery of the handheld grip 600or pass-through power to the gimbal 100 or camera 120. The handheld gripcan also transmit executable instructions to the camera 120. Theseinstructions can take the form of commands which are sent to the camera120 responsive to a user pressing a button on the handheld grip 600.

In some embodiments, the handheld grip includes a plurality of buttons605, 610, 615, 620, 625. An instruction can be sent from the handheldgrip 600 to the camera 120 responsive to pressing a button. In anembodiment, at least one of the buttons 605, 610, 615, 620, 625 areaccessible from both the front and back of the handheld grip 600 so thatthe user can use the buttons 605, 610, 615, 620, 625 to change cameraparameters while not recording. In one embodiment, a first button 605takes a picture or a burst of pictures. The first button 605 can alsobegin recording a video or terminate the recording of a video if it iscurrently recording. In some embodiments, the camera 120 can be in apicture mode, in which it takes pictures or bursts of pictures, or avideo mode, in which it records video. The result of pressing the firstbutton 605 can be determined by whether the camera 120 is in video modeor camera mode. A second button 610 can toggle the mode of the camera120 between the video mode and picture mode. A third button 615 cantoggle the power of the handheld grip 600, the camera 120, and thegimbal 100 (e.g., in a single press to power on all devicessimultaneously). In one embodiment, the gimbal 100 can detect whether ornot a camera 120 is attached to it and will not turn on after pressingthe power button 615 unless a camera 120 is attached to it. A fourthbutton 620 can change the mode of the camera 120 so that it takes burstsof pictures rather than a single picture responsive to pressing thefirst button 605. A fifth button 625 can change the frame rate at whichthe camera 120 records videos. In some embodiments, a button on thehandheld grip can also change the resolution or compression rate atwhich pictures or videos are recorded. The handheld grip can includelight emitting diodes (LEDs) or other visual indicators which canindicate the mode that the camera is operating in. For example, an LEDof a first color can be turned on in order to indicate that the camera120 is in picture mode and an LED of a second color can be turned on toindicate that the camera 120 is in video mode. Additionally, the LEDsmay indicate a power status, charging status, when the battery isdepleted, error states (e.g., that calibration is suggested), etc. Inone embodiment, the status indicators on the grip 600 operate accordingto a same pattern as that on a remote control device described infurther detail below to provide ease of operation for a user whenswitching the camera 120 and gimbal 100 between different mountingplatforms 110. Additionally, operating the buttons 605, 610, 615, 620,625 on the handheld grip 600 can cause a display on the camera 120 tochange (e.g., on a rear display screen) in order to notify the user ofthe changing camera settings. Furthermore, the handheld grip 600 maysend information to display on the camera, such as, for example, anerror state indicating that the gimbal 100 should be re-calibrated. Inone embodiment, the gimbal 100 may first attempt a self-recalibrationand only alert the user if the self-recalibration fails.

In one embodiment, the handheld grip 600 has only two buttons: a firstbutton 605 which operates generally as a shutter button, and a secondbutton 610 which provides a number of different functions including, forexample, power on/off, mode change, inserting a metadata tag, andproviding a reset. For example, in a particular configuration, a shortpress of the second button 610 (e.g., less than 5 seconds) when the grip600 is off simultaneously turns on the grip 600, the gimbal 100, and thecamera 120. Alternatively, a long press of the second button 610 (e.g.,more than 5 seconds) when the grip is initially off turns on the grip600, the gimbal 100, and the camera 120 in a calibration mode (e.g., areset). When the grip 600 is on but not recording video, a short pressof the second button 610 may operate to change between operating modesof the camera 120 (e.g., single photo, burst photo, time lapse photo,video, etc.) while a long press simultaneously powers off the grip 600,the camera 120, and the gimbal 100. Furthermore, pressing the secondbutton 610 while the camera 120 is recording video may instructs thecamera 120 to insert a metadata tag in a recorded video, where themetadata tag can specify the time at which the second button 610 waspressed. This feature enables the user to easily highlight a moment inthe captured video and later identify the highlighted moment for replayand/or editing.

In the two button embodiment, the first button 605 may operate generallyas a shutter button. For example, when the grip 600, gimbal 100, andcamera 120 are turned on, a short press of the first button 605 maycause the camera 120 to start or stop recording video or take aphotograph, depending on the current camera/video mode. Furthermore, along press of the first button 605 when the grip 600, gimbal 100, andcamera 120 are turned on may cycle between gimbal modes as discussedbelow. Additionally, in one embodiment, a long press of the first button605 when the grip 600, gimbal 100, and camera 120 are off may cause thegrip 600, gimbal 100, and camera 120 to turn on and automatically begincapturing video or take a photograph with a single press.

Additionally, the buttons on the handheld grip 600 or on the camera 120may be used to configure other aspects of the gimbal 100 such as, forexample, a stabilization mode (discussed further below), a pitchvelocity, pitch position, yaw velocity, yaw position, update frame IMUdata, status information, or firmware updates.

In some embodiments, the handheld grip 600 can include an audio outputdevice, such as an electroacoustic transducer, which plays a soundresponsive to pressing a button. The sound played by the audio outputdevice can vary depending on the mode of the camera. By way of example,the sound that is played when a video recording is initiated isdifferent than the sound that is played when a picture is taken. As willbe known to one skilled in the art, additional buttons with additionalfunctions can be added to the handheld grip 600 and some or all of theaforementioned buttons can be omitted.

In some embodiments, the rotational angle of the camera 120 to whicheach motor corresponds can vary depending on the mount platform 110 towhich the gimbal 100 is attached. In the embodiment shown in FIG. 6, thefirst motor 301 controls the yaw of the camera 120, the second motor 302(not shown in FIG. 6) controls the roll of the camera 120, and the thirdmotor 303 controls the pitch of the camera 120. This configurationdiffers from that in FIG. 3A and FIG. 3B which depict the motorscontrolling the roll, yaw, and pitch, respectively. In some embodiments,the same gimbal 100 can operate in both configurations, responsive tothe mount platform 110 to which it is connected. For example, whenconnected to the handheld grip 600 the gimbal's motors can operate asyaw, roll, and pitch motors, respectively, and when connected to theaerial vehicle 200 the gimbal's motors can operate as roll, yaw, andpitch motors.

In some embodiments, the camera's rotation for each axis of rotation canbe fixed or unfixed. When the camera's rotation is fixed on an axis, thegimbal 100 will operate to ensure that the camera will maintain thatsame orientation (or approximately so), relative to a reference (e.g.,the horizon or user-defined heading), on that axis despite the movementof the handheld grip. Conversely, when the rotation of the camera 120 isunfixed on an axis, then the camera's rotation on that axis can changewhen the handheld grip 600 is rotated. For example, if the yaw of thecamera 120 is unfixed then a change in the yaw of the handheld grip 600by ϕ degrees can correspond to a change in the yaw of the camera 120 byϕ or −ϕ degrees (depending on the point of reference for which the yawis considered). If all three of the camera's axes are unfixed, then themotors 301, 302, 303 of the gimbal 100 will remain fixed (i.e., theywill not turn) when the handheld grip 600 changes orientation. Thegimbal control system 150 can have a fixed yaw mode and an unfixed yawmode which dictates that the yaw of the camera 120 should remain fixedor unfixed, respectively. Similarly the gimbal control system 150 canhave a fixed and unfixed mode for the roll and the pitch. The user canset the mode to unfixed for a certain axis and reorient the camera 120to the desired angle along that axis, then set the mode for the axis tofixed so the camera 120 will remain at that angle. This will allow auser to easily set the preferred angle of the camera relative to theground. The gimbal control system 150 can still stabilize the rotationalong an axis, while in unfixed mode. In one embodiment, a second button610 toggles the yaw mode between fixed and unfixed, the third button 615toggles the pitch mode between fixed and unfixed, and the forth button620 toggles the roll mode between fixed and unfixed. The axes of thegimbal 100 can be in a fixed mode or unfixed mode while connected to theaerial vehicle 200, as well.

In one embodiment, three selectable stabilization modes are enabled. Ina first stabilization mode (e.g., a yaw “follow” mode), the yaw isunfixed and the pitch and roll are fixed relative to a horizonline-of-sight. In this embodiment, the yaw will be roughly fixed in thesame direction relative to the mount device so that camera rotates asthe user rotates the handheld grip 600, thus enabling the camera 110 tocapture a horizontal pan across a scene with the pitch and roll willremain fixed relative to a horizontal plane (e.g., the ground or horizonline-of-sight). In one embodiment, instead of the yaw being completelyunfixed, the rotation may be dampened by the gimbal 100 applying a smallcounteracting force in order to smooth the panning motion. In oneembodiment, this first stabilization mode may be a default mode.

In a second stabilization mode (e.g., a tracking “locked” mode), theroll may be fixed (i.e., stabilized) relative to a horizonline-of-sight, the pitch is fixed (i.e., stabilized) at a user-definedangle which is set at a user-defined pitch when the mode is activated,and the yaw may be fixed (i.e. stabilized) at a user-defined heading setwhen the mode is activated. This mode may enable the user to lock thecamera onto a particular user defined location while maintaining thecamera roll level to the ground.

In a third stabilization mode, (e.g., a yaw and pitch follow mode), theroll may be fixed (i.e. stabilized) relative to a horizon line-of-sight,the pitch is unfixed but may optionally be dampened to smoothly followthe vertical orientation of the handheld grip 600, and the yaw may beunfixed but may optionally be dampened to smoothly follow the horizontalorientation of the handheld grip 600. This mode thus may enable the userto stabilize the camera roll level to ground, but allowing the user tofollow a moving object with the camera 120.

In an embodiment, the handheld grip 600 may be constructed of a highlydurable a wear-resistant material for surface finishing.

Example Aerial Vehicle System

FIG. 7 illustrates a gimbal 100 attached to a remote controlled aerialvehicle 200, which communicates with a remote controller 720 via awireless network 725. The remote controlled aerial vehicle 200 in thisexample is shown with a housing 230 and arms 235 of an arm assembly. Inaddition, this example embodiment shows a thrust motor 240 coupled withthe end of each arm 130 of the arm assembly. Each thrust motor 240 maybe coupled to a propeller 710. The thrust motors 240 may spin thepropellers 710 when the motors are operational. The gimbal 100mechanically connects a camera 120 to the remote controlled aerialvehicle 200 and may also provide an electrical communication pathbetween the camera 120 and the remote controlled aerial vehicle 200.

The aerial vehicle 200 may communicate with the remote controller 720through the wireless network 725. The remote controller 725 can be adedicated remote controller or can be another computing device such as alaptop, smartphone, or tablet that is configured to wirelesslycommunicate with and control the aerial vehicle 200. In one embodiment,the wireless network 725 can be a long range Wi-Fi system. It also caninclude or be another wireless communication system, for example, onebased on long term evolution (LTE), 3G, 4G, or 5G mobile communicationstandards. In place of a single wireless network 725, the unidirectionalRC channel can be used for communication of controls from the remotecontroller 720 to the aerial vehicle 200 and a separate unidirectionalchannel can be used for video downlink from the aerial vehicle 200 tothe remote controller 720 (or to a video receiver where direct videoconnection may be desired).

The remote controller 720 in this example includes a first control panel750 and a second control panel 755, an ignition button 760, a returnbutton 765 and a display 770. A first control panel, e.g., 750, can beused to control “up-down” direction (e.g. lift and landing) of theaerial vehicle 200. A second control panel, e.g., 755, can be used tocontrol “forward-reverse” direction of the aerial vehicle 200. Eachcontrol panel 750, 755 can be structurally configured as a joystickcontroller and/or touch pad controller. The ignition button 760 can beused to start the rotary assembly (e.g., start the propellers 710). Thereturn button 765 can be used to override the controls of the remotecontroller 720 and transmit instructions to the aerial vehicle 200 toreturn to a predefined location as further described herein. Theignition button 760 and the return button 765 can be mechanical and/orsolid state press sensitive buttons. In addition, each button may beilluminated with one or more light emitting diodes (LED) to provideadditional details. For example the LED can switch from one visual stateto another to indicate with respect to the ignition button 760 whetherthe aerial vehicle 200 is ready to fly (e.g., lit green) or not (e.g.,lit red) or whether the aerial vehicle 200 is now in an override mode onreturn path (e.g., lit yellow) or not (e.g., lit red). The remotecontroller 720 can include other dedicated hardware buttons and switchesand those buttons and switches may be solid state buttons and switches.The remote controller 720 can also include hardware buttons or othercontrols that control the gimbal 100. The remote control can allow itsuser to change the preferred orientation of the camera 120. In someembodiments, the preferred orientation of the camera 120 can be setrelative to the angle of the aerial vehicle 200. In another embodiment,the preferred orientation of the camera 120 can be set relative to theground.

The remote controller 720 also includes a screen (or display) 770 whichprovides for visual display. The screen 770 can be a touch sensitivescreen. The screen 770 also can be, for example, a liquid crystaldisplay (LCD), an LED display, an organic LED (OLED) display or a plasmascreen. The screen 770 can allow for display of information related tothe remote controller 720, such as menus for configuring the remotecontroller 720 or remotely configuring the aerial vehicle 200. Thescreen 770 also can display images or video captured from the camera 120coupled with the aerial vehicle 200, wherein the images and video aretransmitted via the wireless network 725. The video content displayed byon the screen 770 can be a live feed of the video or a portion of thevideo captured by the camera 120. For example, the video contentdisplayed on the screen 770 is presented within a short time (preferablyfractions of a second) of being captured by the camera 120. In someembodiments, the layout of the visual display is adjusted based on thecamera 120 connected to the gimbal 100. For example, if the camera 120is not capable of providing a live feed of captured video, the visualdisplay layout may be adjusted to omit a panel for display of the livecamera feed, whereas otherwise the life feed would be displayed.

The video may be overlaid and/or augmented with other data from theaerial vehicle 200 such as the telemetric data from a telemetricsubsystem of the aerial vehicle 200. The telemetric subsystem mayinclude navigational components, such as a gyroscope, an accelerometer,a compass, a global positioning system (GPS) and/or a barometric sensor.In one example embodiment, the aerial vehicle 200 can incorporate thetelemetric data with video that is transmitted back to the remotecontroller 720 in real time. The received telemetric data can beextracted from the video data stream and incorporate into predefinetemplates for display with the video on the screen 170 of the remotecontroller 720. The telemetric data also may be transmitted separatefrom the video from the aerial vehicle 200 to the remote controller 720.Synchronization methods such as time and/or location information can beused to synchronize the telemetric data with the video at the remotecontroller 720. This example configuration allows a user, e.g.,operator, of the remote controller 720 to see where the aerial vehicle200 is flying along with corresponding telemetric data associated withthe aerial vehicle 200 at that point in the flight. Further, if the useris not interested in telemetric data being displayed real-time, the datacan still be received and later applied for playback with the templatesapplied to the video.

Additionally, error conditions or other flight status information may betransferred form the aerial vehicle 200 to the remote controller 720 fordisplay.

The predefine templates can correspond with “gauges” that provide avisual representation of speed, altitude, and charts, e.g., as aspeedometer, altitude chart, and a terrain map. The populated templates,which may appear as gauges on screen 170 of the remote controller 720,can further be shared, e.g., via social media, and or saved for laterretrieval and use. For example, a user may share a gauge with anotheruser by selecting a gauge (or a set of gauges) for export. Export can beinitiated by clicking the appropriate export button, or a drag and dropof the gauge(s). A file with a predefined extension will be created atthe desired location. The gauge to be selected and be structured with aruntime version of the gauge or can play the gauge back through softwarethat can read the file extension.

In one embodiment, when the gimbal 100 is connected to the aerialvehicle 200 it is automatically configured to operate in a fourth modein which the roll is stabilized at a horizon line-of-sight, the pitch isstabilized at a user-defined angle set using the remote controller 720,and the yaw is unfixed (but may optionally be dampened to smoothrotation) to enable to follow the orientation of the aerial vehicle 200.In another embodiment, a mode may be available in which the yaw is alsostabilized. In this embodiment, the gimbal 100 may receive flightcontroller information from the aerial vehicle 200 to enable it toproactively stabilize against impending yaw movements of the aerialvehicle 200. In another embodiment, the gimbal 100 itself may determinethe motion of the aerial vehicle 200 to enable it to stabilize.

Dampening Connection

FIGS. 8A and 8B show an example of a dampening connection, which can bea connection between the gimbal 100 and a mount platform 110, such asthe aerial vehicle 200. The dampening connection can include a floatingconnection base 800, a locking cylindrical shell 810, a plurality ofelastic pillars 820 (e.g., four elastic pillars 820), a connectionhousing 830, a plurality of tapered locking blocks 840 (e.g., fourtapered locking blocks 840), a fixed mount floor 850 with four slots855, and a fixed mount ceiling 860. The fixed mount floor 850 and thefixed mount ceiling 860 may rigidly attach to the mount platform 110such that a gap exists between a top surface of the fixed mount floor850 and a bottom surface of the fixed mount ceiling 860. The fixed mountfloor 850 may furthermore include a side surface adjacent to thefloating connection base 800. The locking cylindrical shell 810 mayattach to the connection housing 830, and may be capable of beingrotated, which can be used to lock the attachment portion 350 of thegimbal 100 into the connection housing 830. The connection housing 830may attach (e.g., rigidly) to the floating connection base 800. Thefloating connection base 800 may attach to the fixed mount ceiling 860by the four elastic pillars 820. For example, the elastic pillars 820may protrude from a bottom surface of the fixed mount ceiling 860 andmay mechanically couple to a top surface of the floating connection base800 such that the floating connection base 800 hangs below the fixedmount ceiling 860 and adjacent to the fixed mount floor 850. Thefloating connection base 800 may have a plurality of tapered lockingblocks 840 projecting out of it towards the fixed mount floor 850. Forexample, the tapered locking blocks 840 may protrude from the floatingconnection base 800 in a direction substantially perpendicular to theplurality of elastic pillars 820. In this embodiment, each of thetapered locking blocks 840 has a corresponding slot 855 into which itfits. The corresponding slots 855 may be bored into the fixed mountfloor 850 to form a cavity reciprocal to the locking blocks 840. Thetapered locking blocks 840 may be tapered such that a first end attachedto the floating connection base 800 has a greater diameter than a secondend protruding into or adjacent to the locking slots 855.

Compared to a rigid mechanical connection, the dampening connection canhelp to dissipate high frequency vibrations in the gimbal 100 and toprevent, to some degree, the gimbal 100 from vibrating, for example,when the aerial vehicle 200 is operational. The dampening may operateboth to enable the camera 120 to capture more stable video and also toavoid audible artifacts from vibrations of the camera 120, gimbal 100,or mounting platform 110. The dampening connection depicted in FIGS. 8Aand 8B is a mechanical connection between the gimbal 100 and the aerialvehicle 200, but similar structures can be used to connect the gimbal100 to other mount platforms, such as ground vehicle, an underwatervehicle, or a handheld grip. FIG. 8A shows a vertical perspective(looking down) of the dampening connection, wherein the fixed mountceiling 860 has been removed. FIG. 8B shows a horizontal view of thedampening connection. Both FIG. 8A and FIG. 8B are simplified forillustrative purposes, and thus the shapes, relative sizes, and relativepositions of the components of the dampening connection are shown forease of discussion purposes.

The dampening connection can comprise a floating connection base 800,which may be mechanically coupled to four elastic pillars 820. Theelastic pillars 820 may mechanically connect the floating connectionbase 800 to the fixed mount ceiling 860. Aside from the four elasticpillars 820, the floating connection base 800 is not rigidly connectedto the other components of the aerial vehicle 200 in one embodiment,which allows it a small range of motion. The floating connection base800 may be rigidly mechanically coupled to a connection housing 830.When the gimbal 100 is locked into the connection housing 830, theconnection housing 830 may contain the mount connector 304 of the gimbal100. The electronic connector which electrically connects the gimbal 100and the aerial vehicle 200 may be enclosed in the connection housing830.

A locking cylindrical shell 810 may be mechanically connected to theconnection housing 830. The locking cylindrical shell 810 can rotatealong its axis. The user can insert the end of the gimbal 100 into theconnection housing 830 and turn the locking cylindrical shell 810 (e.g.,to a first position) in order to lock the gimbal 100 to the connectionhousing 830. When the gimbal 100 and the connection housing 830 are thuslocked together, the gimbal 100, the connection housing 830, and thefloating connection base 800 are all rigidly connected together. A usercan unlock the gimbal 100 from the connection housing 830 by twistingthe locking cylindrical shell 810 in the opposite direction (e.g., to asecond position), which will allow the user to remove the gimbal 100from the connection housing 830.

When a contact force is exerted on the gimbal 100 by the user in orderto insert the mount connector 304 into the connection housing 830, thefloating connection base 800 will be pushed backwards (e.g., in FIGS. 8Aand 8B, the force would be directed to the left). This may cause adeformation of the elastic pillars 820 due to a shearing force, and thetapered locking blocks 840 may be forced into the corresponding slots855 on the fixed mount floor 850. For example, when a net contact forceis applied to the floating connection base 800 in a direction towardsthe fixed mount floor 850, first ends of the elastic pillars 820attached to the floating connection base 800 may be displaced relativeto second ends of the elastic pillars attached to the fixed mountceiling 860. The elastic pillars 820 may be mechanically coupled to thefloating connection base 800 and to the fixed mount ceiling 860, and, inthe absence of a shearing force, hold the floating connection base 800at an equilibrium position, relative to the fixed mount ceiling 860,which is rigidly mechanically coupled to the chassis of the aerialvehicle 200. The fixed mount floor 820 may also be rigidly mechanicallycoupled to the chassis of the aerial vehicle 200. In some embodiments,the fixed mount floor 820 and the fixed mount ceiling 860 are conjoined.

In one embodiment, at equilibrium (e.g., when the user is not applying acontact force on the gimbal 100), the tapered locking blocks 840 areheld, by shear forces on the elastic pillars 820, at a position that isnot flush with the corresponding slots 855. In other words, a gap existsbetween ends of the locking blocks 840 and corresponding ends of theplurality of slots 855. The gap between the tapered locking blocks 840and their corresponding slots 855 can be small (e.g., 2-5 millimeters).In some embodiments, at equilibrium, the tapered locking blocks 840 restoutside the corresponding slots 855. When a force pushes the taperedlocking blocks 840 into the slots 855 on the fixed mount floor 850, thefloating connection base 830 can be locked in place, which can make iteasier for the user to turn the locking cylindrical shell 810. Forexample, when a net contact force is applied in a direction to thefloating connection base 830 in a direction towards the fixed mountfloor 850, the ends of the locking blocks 840 can be flush withcorresponding ends of the locking slots 855. Once the user is no longerpushing on the gimbal 100, the restoring sheer force on the elasticpillars 820 can move the floating connection base 830 back into itsequilibrium position. In this equilibrium position, the floatingconnection base 830 has some freedom of movement, which can have theresult of dampening oscillations on the gimbal 100 or the aerial vehicle200. Thus, when connected to the aerial vehicle 200, the gimbal 110 can“float” (i.e., is not rigidly coupled to the aerial vehicle 200) duringnormal operation.

Rotating Platform

FIG. 9 illustrates an example embodiment of a gimbal 100 coupled to arotating platform 900. The rotating platform 900 may include a base 910and a rotating gimbal mount 920. The mount connector 310 of the gimbal100 may couple to a reciprocal coupling end of the rotating gimbal mount920. The base 910 may contain a motor which rotates a shaft, which iscoupled to the rotating gimbal mount 920. Operation of this motor can becontrolled by control logic in the rotating platform 900. The motor inthe base 910 can be used to rotate the gimbal 100 and the camera 120,thus facilitating panning of the camera 120 or tracking of an object.The camera 120 in FIG. 9 is depicted with a pitch of about 45° upwards.

In an embodiment, the rotating gimbal mount 920 rotates relative to thebase 910, which in turn rotates the gimbal 100 and the camera 120. Inthis example configuration, when the base 910 is level relative to theground, rotation of the rotating gimbal mount 920 adjusts the yaw of thecamera 120. If the first motor 301 of the gimbal 100 is not able torotate continuously (e.g., the first motor 301 is restricted to acertain range of angles) the rotating platform 900 can be used tocontinuously rotate the camera 120, whereas otherwise that would not bepossible.

In some embodiments, the motor which rotates the rotating gimbal mount920 and the first motor 301 of the gimbal 100 have the same axis ofrotation. The gimbal control system 150 can utilize both the first motor301 and the motor connected to the rotating gimbal mount 920 inconjunction. The motor connected to rotating gimbal mount 920 can becapable of relatively high torque and speed, but be less precise thanthe first motor 301. The motor connected to rotating gimbal mount 920can be used to provide large rotational velocity and acceleration, whilethe first motor 301 performs comparatively smaller rotations that serveto smooth out the panning of the camera 120.

The base 910 may also contain a battery, with which it provides power tothe gimbal 100 and the camera 120. The base 910 may also connect to anexternal power supply. In some embodiments, the base includes aninterface to receive instructions to perform an action with the camera120 or instructions that specify which object the camera 120 is totrack. The interface may be a physical interface such as buttons,switches, or a touch screen by which input from a user is received. Theinterface can be a communication interface which allows the rotatingplatform 900 to receive instructions from an external device. Thisexternal device can, for example, be a dedicated remote controller or ageneric user device. The communication interface can be a wiredcommunication interface which utilizes protocols such as Ethernet, USB,or HDMI or a wireless communication interface such as a WiFi orBluetooth. The communication interface can be used to receiveinstructions and to transmit images and video captured by the camera 120to the external device. In some embodiments, the base 910 also includesweights, which serve to keep the rotating platform 900 upright andstable. In some embodiments, the base 910 can be directly connected to afloor or wall.

The base 910 of the rotating platform 900 may include a mount platformcontrol logic unit 113. The mount platform control logic unit 113 mayimplement an algorithm in which the rotating gimbal mount 920 is usedfor panning rotation and the motors of the gimbal 301, 302, 303 are usedfor precise movement. For example, if the camera 120 is tracking aquick-moving object, then, in order to track the object, the yaw of thecamera 120 may need to be rotated quickly. The control algorithmimplemented by the mount platform control logic unit 113 can do “broad”tracking, in regards to yaw, with the rotating gimbal mount 920 and“precise” tracking with the first motor 301 of the gimbal 100. In thismanner, a powerful, imprecise motor in the rotating platform 900 and anaccurate, low torque first motor 301 in the gimbal 100 may be used inconjunction to produce a yaw rotation that is quick, which a potentialfor high acceleration, but is still able to track an object precisely.

In some embodiments, the mount platform control logic 113 controls thecamera 120, the motor in the base 910, and/or the gimbal 100 to pan thecamera 120 to take panoramic photos. Capturing a panoramic photo maycomprise capturing multiple images with the camera 120 at differentorientations and stitching the images together with image processingsoftware to generate a single composite panoramic photo. In someembodiments, the panoramic photo comprises a 360° photo. In someembodiments, the mount platform control logic 113 controls the camera120, the motor in the base 910, and/or the gimbal 100 to pan the cameraslowly in order to generate a time lapse video. The frame rate of thetime lapse video may be based on the movement speed of an object trackedby the camera 120. The frame rate may be determined such that thetracked object appears to move smoothly in the video. For example, atime lapse video can be captured of a hiker who is far away from thecamera 120. The mount platform control logic 113 can control the panningand image capture of the camera 120 to capture time lapse video in whichthe hiker moves smoothly without capturing an excessive number offrames. The time lapse may be captured with or without panning thecamera 120.

In one embodiment, the mount platform control logic 113 of the rotatingplatform 900 may communicate with, and be controlled by, a remotecomputing device. For example, an application executing on a mobiledevice or the remote controller used with the aerial vehicle 200 may beused to control operation of the rotating platform 900. Furthermore, inone embodiment, operation of the rotating platform can be controlled viathe attached camera 120. For example, the rotating platform 900 maybegin rotating automatically when the camera 120 is configured tooperate in a panoramic mode and image or video captured is initiated.The camera 120 may itself be controlled by another remote device such asa mobile computing device or remote control, and the camera relaysrelevant instructions to the rotating platform 900 via a connectionthrough the gimbal 100 to control its operation.

In another embodiment, the rotating platform 900 may control one or morefunctions of the camera 120. For example, the rotating platform 900 maycause the camera 120 to begin recording or take a picture when therotating platform 900 begins rotating or when it reaches certain anglesin its rotational path. Furthermore, the rotating platform 900 may causethe camera 120 to configure itself in a particular mode (e.g., apanoramic mode) suitable for operation with the rotating platform. Inone embodiment, control signals from the rotating platform 900 to thecamera 120 may be sent via a wired interface through the gimbal 100. Inanother embodiment, the rotating platform 900 may communicate wirelesslywith the camera 120. In yet another embodiment, the rotating platform900 may communicate with a mobile computing device executing anapplication that in turns controls operation of the camera 120.

FIGS. 10A, 10B, 10C, and 10D are block diagrams that illustrate examplemethods for controlling the motor in the base 910 of the rotatingplatform 900 and one or more motors in the gimbal 100 to control theorientation of the camera 120 coupled to the gimbal 100 to track anobject. The control methods illustrated in FIGS. 10A, 10B, 10C, and 10Dmay be implemented by a combination of the gimbal control system 150 andthe motors in the gimbal 100 and rotating platform 900. The orientationof the camera 120 may be controlled along a specific axis. The methodsshown in FIGS. 10A, 10B, 10C, and 10D may be implemented by the rotatingplatform 900 and gimbal 100 shown in FIG. 9 to control the yaw of thecamera 120. For example, in an embodiment, the control method includesdetecting a position of an object to be tracked relative to anorientation of the camera, determining a desired motion and/or position(e.g., a motion state) of the camera suitable for tracking the objectand current motion and/or position of the camera. The motor of therotating platform, the gimbal are both are then controlled depending onthe current and desired motions and/or positions to reduce a differencebetween the current motion and/or position and the desire motion and/orposition. Herein, the control methods are discussed with respect to thefirst motor 301 of the gimbal 100. However, the control methods might beemployed with respect to other gimbal motors in an embodiment in which adifferent motor configuration to that of gimbal 100 is mounted on therotating platform 900.

The control methods illustrated in FIGS. 10A, 10B, and 10C, and 10D makereference to the angular velocity of the motor of the rotating platform900, ω_(RP), the angular velocity of the first motor 301 of the gimbal100, ω_(G), and the angular velocity of the camera 120, ω. The angularvelocity of a motor denotes the angular velocity of the shaft of themotor relative to the rest of the motor. The angular velocity of thecamera 120, ω, denotes the yaw component angular velocity only. Thus, itshould be evident that ω_(RP)+ω_(G)=ω. The angular velocity of eithermotor (ω_(RP) or ω_(G)) can be detected by, for example, a rotaryencoder coupled to the shaft of the motor or detecting the angularvelocity with a gyroscope or accelerometer. The gimbal control system101 may also change ω_(G) or ω_(RP) by modulating the power delivered tothe corresponding motor or by changing the setpoint of a PID whichcontrols the corresponding motor. The angular velocity of the camera120, ω, may be detected by orientation sensors, accelerometers,gyroscopes, or magnetometers in the camera or the IMU in the gimbal 100.Alternately, the angular velocity of the camera 120, ω, may be detectedby the sensor unit 101 on the gimbal 100. Rather than detecting eachvalue in the set {ω_(RP), ω_(G), ω} directly, one value can be detectedindirectly if the other two have been detected using the relationω_(RP)+ω_(G)=ω (e.g., if ω_(G) and ω are detected, ω_(RP) can becalculated as ω_(RP)=ω−ω_(G)). Each of the described processes may berepeated periodicially or when a change in motion is detected so as tocontinuously control motion of the camera 120 to track the object.

FIG. 10A is a block diagram illustrating a control method for allowingthe camera 120 to track a moving object. In the illustrated examplemethod, the first step of the control method 1000 is detecting 1001 theposition of a tracked object. Types of tracked objects and algorithmssuitable for tracking said objects will be further discussed herein. Theposition of the tracked object may be a target object angular position,θ_(TO), wherein the target object angular position is the angle of thedisplacement direction between the camera 120 and the tracked object.Detecting and locating the tracked object to estimate the tracked objectposition, θ_(TO), may use, for example, an object recognition algorithmsapplied to images captured by the camera 120, one or more GPS receivers,a directional microphone system, or some combination thereof.

In the illustrated example method, the gimbal control system 150calculates 1002 a target angular velocity, ω_(t), for the camera 120.The target angular velocity, ω_(t), may be based on the orientation ofthe camera 120, the displacement direction between the camera 120 andthe tracked object, the present angular velocity of the camera 120, andthe angular velocity of the tracked object (denoted herein as ω_(TO)).The angular velocity of the tracked object, ω_(TO), is defined relativeto the camera 120 and is an estimate of the rate of change of thedirection of the displacement vector between the camera 120 and thetracked object. ω_(TO) may be estimated based on the difference betweenthe current position of the tracked object and the previous position.θ_(TO) and ω_(TO) both relate only to the yaw axis (i.e., to movement ofthe tracked object in the horizontal plane). The target angularvelocity, ω_(t), may be calculated via a control algorithm using thedisplacement direction between the camera 120 and the tracked object asa setpoint for the yaw of the camera 120 and ω_(TO) as the setpoint forthe angular velocity of the camera 120.

In some embodiments, tracking an object from a received sensor input maybe associated with a delay, δ. For example, if an object is tracked withmachine vision algorithms applied to images captured by the camera 120,the machine vision algorithms may take δ seconds to locate the object inan image. To compensate for this delay, θ_(TO) may be calculated fromthe delayed target object angular position, θ_(δ). For example, θ_(TO)may be calculated as θ_(TO)=θ_(δ)+δ*ω_(TO), where ω_(TO) is calculatedby comparing the current value of θ_(δ) to the previously detected valueof θ_(δ).

The gimbal control system 150 checks whether |ω_(t)|≥T_(ω) 1003, where|x| denotes the absolute value of x. T_(ω) is a threshold value for theangular speed of the camera 120. For example, the threshold value T_(ω)may comprise a predefined or dynamically configured target angularvelocity. In alternate embodiments, the detected angular speed of thecamera 120, ω, is compared to a threshold in place of ω_(t). If|ω_(t)|≥T_(ω), ω_(RP) may be adjusted 1005 toward ω_(RP) where ω_(RP)denotes the angular velocity of the motor of the rotating platform 900.Adjusting 1005 ω_(RP) toward ω_(t) comprises detecting the present valueof ω_(RP) using, for example, a rotary encoder coupled to the shaft ofthe motor of the rotating platform 900. Adjusting 1005 ω_(RP) towardω_(t) further comprises accelerating or decelerating the motor of therotating platform 900 to match ω_(RP) to ω_(t). ω_(RP) can be matched toω_(t) using a control system, such as a PID controller, with ω_(RP) asthe measured value and ω_(t) as the setpoint of the control system.Furthermore, in one embodiment, ω_(G) (denoting the angular velocity ofthe first motor 301 of the gimbal 100) is also controlled to moreprecisely match ω to ω_(t).

If |ω_(t)| is less than T_(ω), then the gimbal control system 150 maycheck whether ω_(RP)=0 1004, where ω_(RP) denotes the angular velocityof the motor of the rotating platform 900. If ω_(RP)=0 (i.e., if themotor of the rotating platform 900 is stationary), the gimbal controlsystem 150 may adjust 1007 ω_(G) toward ω_(t), where ω_(G) denotes theangular velocity of first motor 301 of the gimbal 100. Otherwise, ifω_(RP)≠0, the gimbal control system may adjust 1006 ω_(G) toward wtwhile decelerating ω_(RP).

The example control method 1000 illustrated by FIG. 10A relies on thefirst motor 301 of the gimbal 100 to maintain the orientation andangular velocity of the camera 120 while the magnitude of the targetangular velocity (i.e., |ω_(t)|) is less than a threshold value, T_(ω).While |ω_(t)|<T_(ω), the motor in the rotating platform 900 isdecelerated until it is stopped. If |ω_(t)|>T_(ω), the gimbal controlsystem 150 relies on the motor of the rotating mount 900 to maintain theorientation and angular velocity of the camera 120 while the first motor301 of the gimbal 100 is used to supplement the motor of the rotatingmount 900 to make tracking more precise. Thus, the control method 1000is a control scheme in which the first motor 101 of the gimbal 100 ispreferentially used when the tracking rotation speed is small, but issupplemented by the motor of the rotating platform 900 to avoid unduestrain on the first motor 101 of the gimbal 100. A control method suchas control method 1000, may be advantageous in embodiments in which themotors of the gimbal 100 are more precise than the motor of the rotatingmount 900. Control method 1000 may also be advantageous in embodimentsin which operation of the motor of the rotating mount 900 producessignificant vibration for the camera 120.

FIG. 10B illustrates a second control method 1020. The control method1020 detects 1001 the position of the tracked object. Based on theposition of the tracked object, the gimbal control system 150 calculates1021 a target angular velocity change, Δω_(t). The target angularvelocity change, Δω_(t), may be calculated as Δω_(t)=ω−ω_(t), where ω isthe current angular velocity of the camera 120, and ω_(t) is the targetangular velocity as discussed in relation to control method 1000.

The gimbal control system 150 compares Δω_(t) to a threshold value,T_(Δωt) to determine if |Δω_(t)|≥T_(Δωt) 1022. If |Δ_(ωt)|<T_(Δωt), thegimbal control system 150 may adjust 1024 the angular velocity of thefirst motor 301 of the gimbal 100, ω_(G), while the angular velocity ofthe rotating platform 900, ω_(RP), remains constant. If|Δω_(t)|≥T_(Δωt), the gimbal control system 150 may adjust 1023 theangular velocity of the motor of the rotating platform 900, ω_(RP). Insome embodiments, ω_(G) is also adjusted when |Δω_(t)|≥T_(Δωt). Thesecond control method 1020 thus keeps the angular velocity of therotating platform 900, ω_(RP), constant while |Δω_(t)| is small andaccelerates the motor of the rotating platform 900 when it is needed tocompensate (i.e., when |Δω_(t)|≥T_(Δωt)). Thus, the second controlmethod 1020 may be used to set a maximum angular acceleration (e.g.,T_(Δωt)) for the first motor 301 of the gimbal 100.

FIG. 10C illustrates a third control method 1030. The third controlmethod 1030 first detects 1001 the position of the tracked object. Basedon the position of the tracked object, the gimbal control system 150calculates 1021 a target angular velocity change, Δω_(t). The angularvelocity of the first motor 301 of the gimbal 100, ω_(G), and Δω_(t) aresummed. This sum is compared to a threshold value, T_(ΔωG), to determineif |ω_(G)+Δω_(t)|≥T_(ΔωG) 1022. If |Δω_(G)+Δω_(t)|<T_(ΔωG), the gimbalcontrol system 150 may adjust 1033 the angular velocity of the firstmotor 301 of the gimbal 100, ω_(G), while the angular velocity of therotating platform 900, ω_(RP), remains constant. If|ω_(G)+Δω_(t)|≥T_(ΔωG), the gimbal control system 150 may adjust 1032the angular velocity of the motor of the rotating platform 900, ω_(RP).In some embodiments, ω_(G) is also adjusted when |ω_(G)+Δω_(t)|<T_(ΔωG).The third control method 1030 thus keeps the angular velocity of therotating platform 900, ω_(RP), constant while |ω_(G)+Δω_(t)| is smalland accelerates the motor of the rotating platform 900 to prevent theω_(G) from becoming too large (i.e., when |ω_(G)+Δω_(t)|≥T_(ΔωG)). Thus,the third control method 1030 can be used to set a maximum angularvelocity (e.g., T_(ΔωG)) for the first motor 301 of the gimbal 100.

FIG. 10D illustrates a fourth control method 1040. This control method1040 detects 1041 the angular position of the tracked object, θ_(TO).The angular position of the tracked object, θ_(TO), may be used tocalculate 1042 the angular velocity of the tracked object, ω_(TO). Theangular velocity of the motor of the rotating platform 900, ω_(RP), maybe adjusted 1043 toward ω_(TO). The angular position of the first motor301 of the gimbal 100, θ_(G), may be adjusted toward θ_(G) θ_(TO)−θ,where θ denotes the current angular position of the camera 120. It isnoted that θ_(G) θ_(TO)−θ=θ_(TO)−θ_(RP). The control method 1040 maycompensate for the acceleration or the deceleration of the motor of therotating platform 900 when adjusting 1044 θ_(G) toward θ_(G)+θ_(TO)−θ.Accordingly, the orientation of the camera, θ, is adjusted towardθ_(TO). Thus, in the fourth control method 1040, the rotating platform900 may be used to track the movement in the tracked object and thegimbal 100 corrects drift to fix the orientation of the camera 120, θ,to the angular position of the tracked object, θ_(TO). The controlmethod 1040 tracks the tracked object by implementing “broad” trackingwith the rotating mount platform 900 and “precise” tracking with thefirst motor 301 of the gimbal 100.

Pole Mount Apparatus

FIG. 11 illustrates an example embodiment of a gimbal 100 coupled to apole mount apparatus 1100. The pole mount apparatus 1100 consists of anupper clamp 1110, a lower clamp 1120, a controller 1130, and a cable1140. The two clamps 1110, 1120 are removably coupled to a pole 1150.The cable 1140 electrically connects the upper and lower clamps 1110,1120. The upper clamp 1110 comprises a connection housing 1111, an outershell 1112, and an inner shell 1113. The gimbal 100 can be removablycoupled to the upper clamp 1110. The mount connector 304 of the gimbal100 couples to a reciprocal coupling end in the connection housing 1111.The lower clamp 1120 can be coupled to the controller 1130.

In some embodiments, the upper clamp 1110 is equipped with at least oneelectric motor, which rotates the connection housing 1111 and the outershell 1112 about the axis of the pole 1150. The inner shell 1113 mayremain rigidly coupled to the pole while the outer shell 1112 rotates.In this configuration, the gimbal 100 can continuously rotate about thepole 1150 without twisting the cable 1140 connecting the upper clamp1110 to the lower clamp 1120, which is coupled to the inner shell 1113.In alternate embodiments, the upper clamp 1110 comprises a single shelland when locked to the pole 1150 cannot rotate.

In some embodiments, the clamps 1110, 1120 are not removable from thepole 1150. As such embodiments, the two clamps 1110, 1120 can be lockedonto the pole 1150, which prevents them from being moved. The clamps1110, 1120 can be unlocked which allows them to slide up and down thepole 1150, but not detached from the pole 1150. In some embodiments, theclamps 1110, 1120 can also be rotated around the pole 1150 whenunlocked. In some embodiments, the lower clamp 1120 is rigidly coupledto the pole 1150 and cannot be unlocked, shifted vertically, or rotatedwithout the use of tools. In alternate embodiments, the lower clamp 1120is omitted entirely and the controller 1130 is connected directly to thepole 1150. In some embodiments, the upper clamp 1110 is omitted and themount connector 304 of the gimbal 100 couples directly to acorresponding connector on the pole 1150. In some embodiments, theheight of the pole 1150 is adjustable.

In some embodiments, at least one of the clamps 1110, 1120 has a firstlocking mechanism which enables the clamp to move up and down the pole1150 or to detach from the pole 1150 entirely and a second lockingmechanism which enables the clamp to be rotated about the pole 1150. Theupper clamp 1110 can have a locking mechanism which when locked orunlocked, serves to fix the rotation of the outer shell 1112 or allowfor rotation of the outer shell 1112, respectively. In some embodiments,the lower clamp 1120 is always free to rotate. In some embodiments, theclamps 1110, 1120 are capable of coupling to poles having a range ofthicknesses.

The controller 1130 allows for user input to the control the operationof the camera 120, the gimbal 100, or the rotation of the outer shell1110. The controller 1130 may include a display that provides fordisplay of video or images captured by the camera 120. The controller1130 can receive an input from a user through buttons, switches, or atouch screen and transmit an instruction to the camera 120 to perform anaction. This can be an instruction to take a picture or a burst ofpictures, begin recording a video, terminate the recording of a video,toggle the mode of the camera 120 between a video mode and a picturemode, toggle the power of the camera 120, change the mode of the camera120 so that it takes bursts of pictures rather than a single picture,change the frame rate at which the camera 120 records videos, change theresolution or compression rate at which pictures or videos are recorded.The controller 1130 can also receive input from a user to trigger thegimbal 100 or upper clamp 1110 to perform an action. For example, afterreceiving an input from a user, the controller 1130 can transmit acommand to the gimbal 100 to change the orientation of the camera 120,or transmit a command to the upper clamp 1110 to rotate. In someembodiments the controller 1130 receives power from an internal batteryor an external power source and provides power through the cable 1140 tothe gimbal 100, the motor of the upper clamp 1110, or the camera 120. Insome embodiments, the controller 1130 contains a mount platform controllogic unit 113 which is part of the gimbal control system 150 whichcontrols the movement of the gimbal 100. The control logic 114 mayimplement a tracking method, such as control methods 1000, 1020, 1030,1040. It is noted that control methods 1000, 1020, 1030, 1040 may beimplemented using a combination of the second and third motors 302, 303of the gimbal 100 and an electric motor in the upper clamp 1110 to trackthe horizontal movement of a tracked object.

Unlike the handheld grip 600 or aerial vehicle 200, the pole mountapparatus 1100 is not expected to move. Consequently, the gimbal controlsystem 150 can leave the roll of the camera 120 fixed, rather thancontinuously parsing data from the sensor unit 101 of the gimbal 100 inorder to detect changes. If the gimbal 100 is not actively tracking anobject, then it may be advantageous to fix all of the motors 301, 302,303 of the gimbal 100. Alternately, the gimbal control system 150 canoperate using reduced complexity or with a lower frequency of receivinginput from the sensing unit 101. These simplifications can result inreduced computational complexity and power consumption for the gimbalcontrol systems 150.

In some embodiments, the cable 1140 provides a wired connection whichallows for communication between the controller 1130 and the gimbal 100or the camera 120. The cable 1140 can transmit commands input by a userinto the controller 1130 to the gimbal 100, the camera 120, or the upperclamp 1110. The controller 1130 may also receive captured images orvideo from the camera 120 through the cable 1140. A gimbal control logicunit 102 and a sensor unit 101 on the gimbal 100 can communicate throughthe cable 1140 with control logic unit 13 on the controller 1130 inorder to provide for control of the gimbal 100. In some embodiments, thecable is internal to the pole 1150. In yet other embodiments, the cable1140 could be replaced with a wireless communication connection, e.g.,Bluetooth.

In some embodiments, the cable 1140 retracts into the upper clamp 1110or lower clamp 1120. For example, a button on the lower clamp 1120 cancause the cable 1140 to be automatically retracted into the lower clamp.In this manner a user can easily mitigate excess cable slack.

In some embodiments, the cable 1140 is omitted and the controllercommunicates wirelessly with the gimbal 100 or the camera 120. In someembodiments, the controller 1130 is not attached to the rest of the polemount apparatus 1100, and function as a wireless remote controller. Insome embodiments, the controller 1130 includes a network interface whichallows for communication with a network such as a Wi-Fi network. Thecontroller 1130 may receive commands or transmit images and video overthe network to a second device.

Tracking Objects

In some embodiments, the preferred orientation of the camera 120 isdefined using one or more tracking algorithms, which can be used totrack an object. Tracking can be done via a machine vision algorithmusing images captured by the camera 120, where the machine visionalgorithm identifies and locates the object in the captured images. Inthis case, there is a conversion from a camera reference frame (e.g.,the camera 120), to host (e.g., the aerial vehicle) reference frame. Thegimbal 100 is given a setpoint in the host reference frame such that thetracked point is in camera view (e.g., with respect to the camera 120used for video).

Tracking can also be done via a GPS receiver, wherein the GPS receiveris tracked by the camera 120. If a user is carrying a GPS enabledtracker or similar localization device, the user location will mostlikely be in an earth (global) reference frame. The gimbal 100 setpointis in a local (e.g., that of the mount platform 110) reference frame.The mount platform 110 can have a navigation module that combinesseveral sensors to calculate its own position in a global referenceframe. The mount platform 110 converts user coordinates (e.g., globalreference frame) into a gimbal setpoint (e.g., local reference frame)such that the object is in the view.

GPS tracking is in general, only accurate to within a few meters.Consequently, GPS tracking can be used in conjunction with another formof tracking to provide for more accurate tracking. In one embodiment,information from a GPS tracker and a GPS receiver on a mount platform110 can be used to provide an estimated position of the GPS tracker,relative to the mount platform 110. A processor on the mount platform110 or the gimbal 100 can calculate a range of angles from the camera,in which the tracked object can be expected to be. This range of anglescan be calculated based on an a priori estimate of GPS accuracy, aquality metric for the GPS signals received, and the distance betweenthe mount platform 110 and the GPS tracker. This range of anglescorresponds to a certain area of an image captured by a camera. Amachine vision system can parse an image or set of images received fromthe camera 120. By limiting the machine vision algorithm to the areadetermined based on the range of angles, the computation expenditureneeded to detect the object with the machine vision system can bereduced and the accuracy can be improved. In another embodiment, themachine vision system examines the entire image received from the camera120, but uses the GPS information in a probabilistically method todetermine which of several candidate objects to track. Combining GPS andmachine vision tracking can achieve better performance than eithersystem in isolation.

A tracked object can also be an audio source, a source radiating anelectromagnetic signal, a device communicatively coupled with the mountplatform 110, or an object identified by a machine vision system. Thetracked object can be detected using appropriate sensors on either thecamera 120 or the mount platform 110, and one or more processors on thecamera 120 or the mount platform 110 calculates the position of thetracked object relative to the mount platform 110. Calculating theposition of the tracked object relative to the mount platform 110 mayinvolve calculating the position of the tracked object relative to thecamera 120 and converting the position to the reference of the mountplatform 110. The position of the tracked object relative to the mountplatform 110 can be used by the gimbal control system 150 to generate asetpoint (e.g., a preferred position) for the gimbal 100, defined sothat the camera 120 is oriented to face the tracked object. The positionof the tracked object relative to the mount platform 110 might be suchthat the camera 120 cannot be oriented to face the tracked object due tothe mechanical limitations of the gimbal 100 or due to the gimbal 100 orthe mount platform 110 obstructing the view of the camera 120. In such asituation, the setpoint of the gimbal 100 can be set to a defaultorientation or the gimbal control system 150 can use a setpoint so thatthe camera 120 is oriented at an orientation as close as possible to theideal orientation.

In some embodiments, the user is able to define a tracked object whichthe camera 120 tracks via a machine vision object tracking algorithm. Avideo feed from the camera 120 or from a camera on the mount platform110 can be transmitted to, for example, a remote controller (e.g., adedicated controller with a display, a smartphone, or a tablet) fordisplay to the user (e.g., on a screen of a remote controller which iscommunicatively coupled to the aerial vehicle 200 coupled to the gimbal100). In addition, through the remote controller the user can select anobject (e.g., by tapping the object on a touchscreen) which selects theobject as the tracked object. A machine vision system can recognize aplurality of objects in the video feed of the camera 120 using an objectclassifier (e.g., a facial recognition system, a classifier configuredto recognize people, or a generic classifier which can be trained torecognize a generic object) and display an indicator on the video feedwhich indicates to the user that the object is available for tracking.

Once a tracked object is selected, a machine vision object trackingalgorithm can be used to orient the camera 120 so that the trackedobject is centered in the frame of the video. The machine visionalgorithms can be used to identify and track objects can be performed byone or more processors on the camera 120, the mount platform 110, aremote controller device communicatively connected to a remotecontrolled vehicle to which the gimbal 100 is mounted, or a remoteserver connected to the mount platform 110 via a network.

The gimbal 100 can also be configured to track an audio source, based onthe directionality of the audio source. The camera 120 or mount platform110 can include a multiplicity of audio receivers (e.g., anacoustic-to-electric transducer or sensor) which can be used to recordsound from an audio source and to estimate the directionality of thesound based on the relative delay between the spatially diverse audioreceivers. The gimbal 100 can track any sound over a certain decibellevel, or with a certain energy within a given frequency range, or thatmatch an audio profile of a user which can be assessed using vocalrecognition algorithms. In an example embodiment, an audio output devicecarried by a user can emit sound at an ultrasonic or infrasonicfrequency (i.e., outside the threshold of human hearing), and this audiooutput device can be tracked by detecting the sound emitted by the audiooutput device. Additionally, the tracked object can be a GPS trackerthat is communicatively coupled to the mount platform 110. The locationdevice can detect its own coordinates via a GPS receiver and transmitthe coordinates to the mount platform 110. The mount platform 110 canthen calculate the position of the GPS tracker relative to itself usinga navigation module that also includes a GPS. In some embodiments, ahandheld remote controller used to control the mount platform 110functions as the GPS tracker.

In some embodiments, a mount platform 110 may include sensors fortracking an object. For example, a mount platform can include lidar,radar, or sonar. Information from a mount platform 110 sensors can beused in conjunction with captured images or video from a camera 120 totrack an object. For example, machine vision algorithms can be used toidentify an object and lidar can be used to continually track theobject. Processing images captured by the camera 120 with machine visionalgorithms can be used to supplement this continuous tracking. Forexample, a machine vision processing system can process one frame inevery N captured video frames in order to determine that the lidarsystem is still correctly tracking the object. Alternately, the machinevision algorithm implemented by the mount platform 110 during continuoustracking may be continuous but may be of lower computational complexitythe algorithm used to initially identify an object. In addition, whenthe lidar tracking system is determined to be unreliable (e.g., whentracking of the object is lost, when the tracked object appears to jumpsuddenly to a new location, when the tracked object appears to quicklychange speeds, when the tracked object appears to change size, etc.),the machine vision algorithm may be again be used to reacquire theobject for tracking.

Each of the aforementioned tracking schemes can allow the camera 120 tocontinuously track an object, such as a user, as the tracked objectmoves around and as the mount platform 110 moves around and rotates. Insome embodiments, multiple tracking schemes can be combined to bettertrack an object. In some embodiments, multiple tracking schemes aresupported by the gimbal control system 150, and the user is able toselect between tracking schemes.

For a mount platform 110 which incorporates motors, such as the rotatingplatform 900 and the pole mount apparatus 1100, tracking can utilize themotor of the mount platform 110 and the motors 301, 302, 303 of thegimbal 100 together. For example, a motor in the pole mount apparatus1100 which rotates the outer shell 1112 of the upper clamp 1110 can beused for large yaw rotations and the motors 301, 302, 303 of the gimbal100 can be used when only small yaw rotations are desired. In someembodiments, the motor of the mount platform 110 can be restricted tolarge, sweeping motions, and the motors of the gimbal motors 301, 302,303 can be used for rapid adjustments to facilitate tracking. Forexample, an upper bound and lower bound can be placed on the rate ofchange of the mount platform's angular velocity.

In some embodiments, the gimbal control system 150 has a plurality ofmodes for determining the behavior of the gimbal 100 which can betoggled by a user. In a center frame tracking mode, an object is trackedso that the object is maintained within the center of the frame of thecamera 120. In an in-frame tracking mode, the orientation of the camera120 remains fixed while a tracked object is within the frame beingcaptured by the camera 120, but when the tracked object is near the edgeof the frame, the orientation of the camera 120 is adjusted to keep theobject within frame. In a contextual tracking mode, there is no specificobject tracked, but once an algorithm identifies an object as an objectof interest with a machine vision algorithm the object will becontinuously tracked. An object can be determined to be an object ofinterest based on movement of the object (e.g., if an object greaterthan a specified size has a speed greater than some upper bound), orbased on an image classifier (i.e., an object classifier whichidentifies people). In a fixed-orientation mode, the orientation of thecamera 120 relative to a reference frame such as the ground inmaintained, but no specific object is tracked. In a scanning mode, thecamera 120 is panned continuously in order to locate an object to track.For example, the rotating mount 900 may continuously rotate so thatobjects within a full 360° can be viewed by the camera 120 andidentified by a machine vision system.

In some embodiments, tracking may control movement of the mount platform110 in addition to that of the gimbal 100. For example, if the gimbal100 is coupled to a mobile mount platform 110, such as an aerial vehicle200 or a ground vehicle, the mount platform 110 may follow the trackedobject. In an example usage case, a gimbal 100 attached to an aerialvehicle 200 is set to follow a user. As the user moves, the aerialvehicle 200 can follow the user so that the user is always within theframe of the camera 120, leaving a certain distance between the aerialvehicle 200 and the tracked user. A motor on a mount platform 110 thatallows for rotation, such as in the rotating platform 900 or the polemount apparatus 1100 can also be used in conjunction with the gimbal 100to track an object.

In some embodiments, vibrations are detected. Vibrations can be detectedby a mechanical vibration sensor (e.g., one or more piezoelectricsensors) or by identifying blur caused by vibration in images capturedby the camera 120. Blur caused by vibrations can be distinguished frommotion blur or rotational blur using digital signal processing. Theaperture size, the shutter speed, and the luminance of the image can beadjusted based on the level of detected vibration. When a large degreeof vibration is detected, the shutter speed can be increased. Theaperture size can be increased or the luminance can be adjusted tocompensate for the decreased exposure resulting from the increase inshutter speed. Increasing the shutter speed can mitigate the effect ofvibration blur, but will result in a more noisy image. In someembodiments, the camera 120 can record video at a higher frame rate inresponse to vibration.

Video Stabilization

FIG. 12 is a block diagram that illustrates an example method forstabilizing the camera 120 with the gimbal 100 mounted on mountplatform, such as aerial vehicle 200. The stabilizing method 1200 usessome combination of Electronic Image Stabilization (EIS), a highresponse gimbal control scheme, and a low response gimbal controlscheme. EIS may be performed by processors on the camera 120, the aerialvehicle 200, or some combination thereof. EIS may involve sequentialframes of a video being captured along with information indicating theorientation of the camera 120. Sensors 440 on the camera 120 may detectthe camera's orientation. In another embodiment, the gimbal 100 maydetect this orientation information with sensors of the sensor unit 101rigidly coupled to the camera 100. The orientation of the camera 120 mayalso be estimated based on rotary encoders in one or more motors (e.g.,301, 302, 303) of the gimbal 100. The stabilizing method 1200 may beperformed by the gimbal control system 150 periodically or continuously(e.g., for every frame of a video).

EIS may involve capturing a frame of video with the camera 120 andcropping the captured frame to a set size (i.e., a set pixel width andheight). The larger frames, captured by the camera, are denoted hereinas source frames and the cropped frames are denoted herein as stabilizedframes. The offset of the cropping may be determined to mitigate theeffect of shifting pitch and yaw orientation of the camera 120 on thevideo. Thus, the sequence of stabilized frames may be shifted so as tocounteract movement of the camera 120. Deviations in the yaw and pitchorientation can be counteracted by proportional horizontal and verticalpixel shifts, respectively. In some embodiments, EIS also corrects fordeviations in the roll of the camera by rotating the source frames tocounteract the roll. The maximum angular deviation that the EIS cancorrect for is limited by the size of the stabilized frames incomparison to the source frames. If the source frame is large comparedto the stabilized frame, EIS can correct for relatively large deviationsin the angular orientation of the camera 120. In some embodiments, thegimbal control system 150 is configured to prevent the camera'sorientation from exceeding a threshold such that the boundaries of astabilized frame do not exceed the boundaries of a source frame. Inalternate embodiments, EIS interpolates between previously capturedframes and the pixels at the edge of a current source frame to generatea stabilized frame with image content that is outside the boundaries ofthe source frame.

In some embodiments, EIS employs a filter on the source frame togenerate the stabilized frame wherein the filter corrects for an imagedistortion due to the geometry of the camera's optics (e.g., an inversefisheye filter for a camera 120 with a fisheye lens). In someembodiments, EIS uses multiple detected instances of the camera'sorientation captured during the exposure period (i.e., the period oftime in which the sensors of the camera 120 are exposed to light) of theframe to correct for motion blur in a frame. The detected “path” of thecamera's orientation during the period of exposure may be used togenerate a pattern which is the convolutional inverse of the blurproduced by the camera's detected movement. This pattern can beconvolved with the source frame to generate the stabilized de-blurredframe. In some embodiments, EIS captures multiple source frames,analyzes the source frame to detect motion blur, and composites the bestframes into a single stabilized frame.

EIS may be performed in real-time or near-real time. In someembodiments, the requisite information to perform EIS (i.e., the sourceframe or a cropped version of the source frame and the detectedorientation) is stored on a memory of the camera 120 or aerial vehicle200. EIS may then be performed during post-processing of the video(e.g., on a user's computer after downloading a file containing thesource video data and orientation information). Real-time EIS typicallyuses less non-volatile storage, but uses more processing power. In someembodiments, EIS is performed in both real-time and post-processing. Thereal-time EIS may involve less computationally expensive operations suchas determining the pixel offset for cropping to generate a partiallystabilized frame. During post-processing, the partially stabilized framemay be rotated to correct for deviations in the roll of the camera 120.The partially stabilized frame may be also be further cropped, filteredto correct for image distortion due to lenses, and the like.

The gimbal control system 150 may select between a high response controlscheme and a low response control scheme. In general, the low responsecontrol scheme does not correct deviations between the ideal orientationof the camera 120 and the detected orientation of the camera 120 asquickly as the high response algorithm. Accordingly, the low responsealgorithm permits more error of the orientation of the camera 120, butgenerally uses less power. FIG. 12 and the corresponding descriptiondescribe high response and low response control schemes, but someembodiments do not include this bifurcation. The division between thetwo control schemes as described herein is intended to illustrate howthe gimbal control system 150 can dynamically adjust control of themovement of the gimbal 100 and EIS to stabilize video while optimizingfor parameters based on the available power, the current energy storedin the battery of the aerial vehicle 200, the internal temperature ofthe aerial vehicle 200, channel capacity between the aerial vehicle 200and the remote controller 720, and processing power available. It willbe apparent to one skilled in the art that the gimbal control system 150may select between more than two control schemes (e.g., a thirdintermediate control scheme between the high response and low responsecontrol schemes). The gimbal control system 150 may also implement asingle control scheme which stabilizes video by dynamically adjustingcontrol of the gimbal 100 and EIS to substantially the same effect asmultiple control schemes.

In one embodiment, the high response control scheme is an underdamped,and the low power control scheme is overdamped or critically damped. Ingeneral, an underdamped control scheme returns the orientation of thecamera 120 to a setpoint (e.g., the angular position of a trackedobject) more quickly than an overdamped or critically damped controlscheme. However, the underdamped control scheme oscillates around asetpoint, whereas an overdamped or critically damped control scheme willnot. In some embodiments, the gimbal control system 150 controls atleast one of the motors of the gimbal 100 with a PID controller and aunderdamped, overdamped, and/or critically damped control scheme isimplemented by setting the weights of the proportional, integral, andderivative components of the PID controller. In alternate embodiments,the high response algorithm has a maximum torque, power, acceleration,and/or angular velocity setting for the motor of the gimbal 100 that ishigher than that of the low response control algorithm.

Returning to FIG. 12, in the example method, the stabilizing method 1200detects 1201 the camera 120 orientation error. In this example, thecamera 120 orientation error is the difference between a targetorientation and a detected orientation of the camera 120. The targetorientation may be based on the position of a tracked object or anequilibrium position defined by the gimbal control system 150. Thecamera orientation error may be checked 1202 against an error thresholdT_(E). Here, the threshold represents a predefined or dynamicallyselected error value. This error threshold T_(E) may be compared againstthe most recently detected camera orientation error, the average cameraorientation error within a period of time, the maximum value of thecamera orientation error within a period of time, and the like.

The error threshold T_(E) may be based on the size of the stabilizedframes in comparison to the source frames. In some embodiments, thesource frames are always the same size for a given camera 120, but thesize of the stabilized video is determined by a user. Larger frames ofthe stabilized video correspond to a smaller error threshold T_(E). Insome embodiments, the error threshold T_(E) is based on the focal lengthof the camera 120 such that the error threshold is small for a camera120 with a large focal length. In some embodiments, the error thresholdT_(E) is based on the shutter speed of the camera 120. A slow shutterspeed corresponds to a low error threshold T_(E). The error thresholdT_(E) may be based on a user configurable setting or a command receivedat the aerial vehicle 200 from the user. For example, the errorthreshold T_(E) may be decreased after receiving a command to capture apicture, capture a burst of pictures, or to start recording video. Theerror threshold T_(E) may be decreased if the aerial vehicle 200 iscurrently using a large amount of power, the current energy stored inthe battery of the aerial vehicle 200 is low, the detected internaltemperature of the aerial vehicle 200 is high, or some combinationthereof.

If the detected orientation error is greater than the error thresholdT_(E), the stabilizing method 1200 may use 1230 a high response gimbalcontrol scheme and real-time EIS. The combination of the high responsegimbal control scheme and EIS can maximize the stabilizationcapabilities of the gimbal 100 and camera, but may also use more powerand processing resources. In some embodiments, the combination of thehigh response gimbal control scheme and EIS is used 1203 when the aerialvehicle 200 performs or is about to perform a maneuver even if thedetected error is not greater than the error threshold T_(E). Such amaneuver may be a change in speed, banking, rotating, changing altitude,landing, lifting off, and the like. Similarly, the combination of thehigh response gimbal control scheme and EIS may also be used 1203whenever the speed, acceleration, angular speed, or angular accelerationof the aerial vehicle 200 is greater than some threshold. In someembodiments, the combination of the high response gimbal control schemeand EIS is used 1203 responsive to an estimation by the aerial vehicle200 of the wind speed or the variance of wind speed.

If the detected orientation error is less than the error thresholdT_(E), the power budget may be checked 1204 against a power budgetthreshold T_(PWR). The power budget may be determined be dynamically bythe gimbal control system 150. The power budget may be low if the aerialvehicle 200 is currently using a large amount of power, the currentenergy stored in the battery of the aerial vehicle 200 is low, thedetected internal temperature of the aerial vehicle 200 is high, or somecombination thereof. The power budget may also be determined by currentpower usage or average power usage with a period of time of the variouscomponents of the gimbal system 160 (e.g., processors or rotors 240 ofthe aerial vehicle 200). The power budget may also be or be determinedby a user-configurable setting. If the power budget is less than thepower budget threshold T_(PWR), then the stabilizing method 1200 usesthe low response gimbal control scheme and real-time EIS. The lowresponse gimbal control scheme uses less power, and the real-time EISmitigates deviations in the camera's orientation. In some embodiments,instead of real-time EIS, video data is stored in a manner suitable forpost-processing EIS, as discussed below in conjunction with FIG. 13. Ifthe power budget is greater than the power budget threshold T_(PWR), thehigh response gimbal control scheme is used 1260 without EIS. The highresponse gimbal control scheme uses more power than the low responsegimbal control scheme, but generally provides for more stable video. Ingeneral, the high response mode produces less motion blur in individualframes than the combination of low response mode and EIS.

In some embodiments, a processing budget check is performed in additionto or instead of the power budget check 1204. The processing budget maybe determined based on the availability of computing resources for theaerial vehicle 200, the camera 200, or both. The processing budget maybe determined based on the current processor utilization, the currentlyavailable random access memory (RAM), the current temperature of one ormore processors, or the length of a queue of operations to be performed.The queue of operations to be performed may include data compressionoperations (e.g., image compression), decoding operations, controlalgorithms, path planning algorithms for determining a flight path forthe aerial vehicle 200, error-correcting coding or decoding, machinevisions algorithms (e.g., object detection), and the like. Theprocessing budget may be checked against a processing budget thresholdT_(PRS). The processing budget threshold T_(PRS) may be based on theprocessing load to perform EIS on one or more frames. If the processingbudget is greater than the processing budget threshold T_(PRS), thegimbal control system 150 may use 1205 the low response gimbal controlscheme and real-time EIS. Conversely, if the processing budget is lessthan the processing budget threshold T_(PRS), the gimbal control system150 may use 1206 the high response gimbal control scheme withoutreal-time EIS. In some embodiments, an EIS algorithm is selected fromamong multiple EIS algorithms, each with different processingrequirements, based on the processing budget or power budget. Forexample, if the processing budget is relatively low, EIS may beperformed without a filtering operation to correct for lens distortion.Metadata may be stored with the frame so that this filtering operationmay be performed in post-processing.

In some embodiments, the camera 120 includes optical image stabilization(OIS). The camera 120 may include OIS is addition to or instead of EIS.The gimbal control system 150 may dynamically control the combination ofOIS, EIS, and movement of the gimbal to stabilize captured video andpictures. For example, OIS may be turned off when the detected errorthreshold is below a certain threshold (e.g., error threshold 1202). Thegimbal control system 150 may also control the aerial vehicle 200 tostabilize the orientation of the camera 120. For example, the aerialvehicle 200 may be controlled to keep the pitch, yaw, and roll within acertain range. The stabilization method 1200 illustrated in FIG. 12 mayalso be employed by a gimbal control system 150 for a gimbal 100attached to mount platforms 110 other than an aerial vehicle.

Additional Considerations

The disclosed configuration describes an electronic gimbal 100 capableof being removably connected to multiple different mount platforms, suchas aerial vehicles, ground vehicles, handheld grips, rotating mounts,and pole mounts. The disclosed configuration further describes anelectronic gimbal 100 capable of removably connecting to multipledifferent cameras and maintaining the orientation of a camera 120 inspace while the mount platform 110 to which the gimbal 100 is attachedchanges orientation. Moreover, the gimbal 100 can contain an internalbus between the camera 120 and the mount platform 110, which providesfor communication. The camera 120 may be removably coupled to adetachable camera frame 130, which, in turn, removably couples to thegimbal 100. The gimbal 100 can also be configured with motors that arenot orthogonal, which provides for a greater viewing angle for thecamera 120. Tracking algorithms can be implemented by the gimbal 100 andmount platform 110 to track on object with the camera 120 attached tothe gimbal. EIS and the movement of the gimbal may be used incombination to stabilize captured images and video.

Also disclosed are mounts to which a gimbal 100 can be removablycoupled. A rotating platform 900 includes a motor which may be used inconjunction with the motors 301, 302, 303 in the gimbal to pan thecamera 120. A pole mount apparatus 1100 couples to a pole 1150 withupper and lower clamps 1110, 1120. The gimbal couples to the upper clamp1110 and a controller 1130 which displays video captured by the camera120 connects to the lower clamp 1120. A handheld grip 600 may couple tothe gimbal 100 and includes a number of buttons to control the operationof the camera 120. An aerial vehicle 200, such as a quadcopter, maycouple the gimbal 100. The aerial vehicle 200 may wirelessly transmitvideo captured by the camera 120 to a remote controller 720. The aerialvehicle 200 may include a dampening base to dissipate high frequencyvibrations in the gimbal 100 and prevent, to some degree, the gimbal 100from vibrating, for example, when the aerial vehicle 200 is operational.

The processes and functions described herein attributed to the gimbal100, camera 120, mount platform 110, pole mount apparatus 1100, aerialvehicle 200, handheld grip 600, or other devices may be implemented viahardware, software, firmware, or a combination of these. In embodimentsdescribed herein, each of the above-named devices may include one ormore processors and one or more non-transitory computer-readable storagemediums. The non-transitory computer-readable storage mediums may storeinstructions executable by one or more of the processors that whenexecuted cause the processor to carry out the processes and functions ofthe respective devices described herein.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied on a machine-readable medium or ina transmission signal) or hardware modules. A hardware module is atangible unit capable of performing certain operations and may beconfigured or arranged in a certain manner. In example embodiments, oneor more computer systems (e.g., a standalone, client or server computersystem) or one or more hardware modules of a computer system (e.g., aprocessor or a group of processors) may be configured by software (e.g.,an application or application portion) as a hardware module thatoperates to perform certain operations as described herein.

In various embodiments, a hardware module may be implementedmechanically or electronically. For example, a hardware module maycomprise dedicated circuitry or logic that is permanently configured(e.g., as a special-purpose processor, such as a field programmable gatearray (FPGA) or an application-specific integrated circuit (ASIC)) toperform certain operations. A hardware module may also compriseprogrammable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that istemporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a hardware modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

The various operations of example methods described herein may beperformed, at least partially, by one or more processors, that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

The one or more processors may also operate to support performance ofthe relevant operations in a “cloud computing” environment or as a“software as a service” (SaaS). For example, at least some of theoperations may be performed by a group of computers (as examples ofmachines including processors), these operations being accessible via anetwork (e.g., the Internet) and via one or more appropriate interfaces(e.g., application program interfaces (APIs).)

The performance of certain of the operations may be distributed amongthe one or more processors, not only residing within a single machine,but deployed across a number of machines. In some example embodiments,the one or more processors or processor-implemented modules may belocated in a single geographic location (e.g., within a homeenvironment, an office environment, or a server farm). In other exampleembodiments, the one or more processors or processor-implemented modulesmay be distributed across a number of geographic locations.

Some portions of this specification are presented in terms of algorithmsor symbolic representations of operations on data stored as bits orbinary digital signals within a machine memory (e.g., a computermemory). These algorithms or symbolic representations are examples oftechniques used by those of ordinary skill in the data processing artsto convey the substance of their work to others skilled in the art. Asused herein, an “algorithm” is a self-consistent sequence of operationsor similar processing leading to a desired result. In this context,algorithms and operations involve physical manipulation of physicalquantities. Typically, but not necessarily, such quantities may take theform of electrical, magnetic, or optical signals capable of beingstored, accessed, transferred, combined, compared, or otherwisemanipulated by a machine. It is convenient at times, principally forreasons of common usage, to refer to such signals using words such as“data,” “content,” “bits,” “values,” “elements,” “symbols,”“characters,” “terms,” “numbers,” “numerals,” or the like. These words,however, are merely convenient labels and are to be associated withappropriate physical quantities.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other. The embodiments are not limited in this context.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the invention. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs for thedisclosed gimbal ecosystem. Thus, while particular embodiments andapplications have been illustrated and described, it is to be understoodthat the disclosed embodiments are not limited to the preciseconstruction and components disclosed herein. Various modifications,changes and variations, which will be apparent to those skilled in theart, may be made in the arrangement, operation and details of the methodand apparatus disclosed herein without departing from the spirit andscope defined in the appended claims.

In additional, additional configurations associated with and/oraugmenting those described herein are provided in Appendix A and B,which are included and incorporated by reference within thisspecification.

What is claimed is:
 1. A method, comprising: detecting an objectrelative to an orientation of a camera coupled to a gimbal; andcontrolling a motor of the gimbal by: determining a target change inangular velocity representing a difference between an angular velocityof the camera and a target angular velocity of the camera suitable fortracking the object; comparing the target change in angular velocity toa threshold change in angular velocity; and responsive to the targetchange in angular velocity not exceeding the threshold change in angularvelocity, adjusting an angular velocity of the motor of the gimbal toreduce the difference between the angular velocity of the camera and thetarget angular velocity of the camera.
 2. The method of claim 1, whereinthe gimbal is coupled to a rotating platform, further comprising:controlling a motor of the rotating platform by: responsive to thetarget change in angular velocity exceeding the threshold change inangular velocity, adjusting an angular velocity of the motor of therotating platform to reduce the difference between the angular velocityof the camera and the target angular velocity of the camera.
 3. Themethod of claim 2, wherein controlling the motor of the gimbal furthercomprises: responsive to the target change in angular velocity notexceeding the threshold change in angular velocity, determining if arotating platform coupled to the gimbal is stationary.
 4. The method ofclaim 3, further comprising: responsive to determining the rotatingplatform coupled to the gimbal is stationary, adjusting the angularvelocity of the motor of the gimbal to reduce the difference between theangular velocity of the camera and the target angular velocity of thecamera.
 5. The method of claim 3, further comprising: responsive todetermining the rotating platform coupled to the gimbal is notstationary, decelerating the rotating platform and adjusting the angularvelocity of the motor of the gimbal to reduce an offset between theangular velocity of the motor of the gimbal and the target angularvelocity of the camera.
 6. The method of claim 2, further comprising:summing an angular velocity of the motor of the gimbal and the targetchange in angular velocity to generate a combined angular velocity;comparing the combined angular velocity to a threshold change in angularvelocity; responsive to the combined angular velocity exceeding thethreshold change in angular velocity, adjusting the angular velocity ofthe motor of the rotating platform to reduce the difference between theangular velocity of the camera and the target angular velocity of thecamera; and responsive to the combined angular velocity not exceedingthe threshold change in angular velocity, adjusting the angular velocityof the motor of the gimbal to reduce the difference between the angularvelocity of the camera and the target angular velocity of the camera. 7.The method of claim 2, further comprising: detecting a first angularposition and a second angular position of the object relative to theorientation of the camera, the first angular position detected at afirst time instance and the second angular position detected at a secondtime instance; determining an angular velocity of the object based onthe first angular position, the second angular position, the first timeinstance, and the second time instance; and adjusting the angularvelocity of the motor of the rotating platform to reduce a differencebetween an angular velocity of the rotating platform and the angularvelocity of the object.
 8. The method of claim 7, further comprising:determining a target angular position of the motor of the gimbal basedon an angular position of the motor of the gimbal, the second angularposition of the object, and an angular position of the camera; andadjusting the angular position of the motor of the gimbal to reduce adifference between the angular position of the motor of the gimbal andthe target angular position of the motor.
 9. A non-transitorycomputer-readable storage medium including instructions that whenexecuted cause a processor to perform operations including: determininga target change in angular velocity representing a difference between anangular velocity of a camera and a target angular velocity of the camerasuitable for tracking an object; comparing the target change in angularvelocity to a threshold change in angular velocity; responsive to thetarget change in angular velocity being below the threshold change inangular velocity, adjusting an angular velocity of a motor of a gimbalcoupled to the camera thereby reducing a difference between the angularvelocity of the camera and the target angular velocity of the camera;and responsive to the target change in angular velocity being above thethreshold change in angular velocity, adjusting an angular velocity of amotor of a rotating platform coupled to the gimbal thereby reducing thedifference between the angular velocity of the camera and the targetangular velocity of the camera.
 10. The non-transitory computer-readablestorage medium of claim 9, further including instructions that whenexecuted cause the processor to perform operations including: responsiveto the target change in angular velocity not exceeding the thresholdchange in angular velocity, determining if the rotating platform isstationary.
 11. The non-transitory computer-readable storage medium ofclaim 10, further including instructions that when executed cause theprocessor to perform operations including: responsive to determining therotating platform is stationary, adjusting the angular velocity of themotor of the gimbal to reduce the difference between the angularvelocity of the camera and the target angular velocity of the camera.12. The non-transitory computer-readable storage medium of claim 10,further including instructions that when executed cause the processor toperform operations including: responsive to determining the rotatingplatform is not stationary, decelerating the rotating platform andadjusting the angular velocity of the motor of the gimbal to reduce anoffset between the angular velocity of the motor of the gimbal and thetarget angular velocity of the camera.
 13. The non-transitorycomputer-readable storage medium of claim 9, further includinginstructions that when executed cause the processor to performoperations including: summing an angular velocity of the motor of thegimbal and the target change in angular velocity to generate a combinedangular velocity; comparing the combined angular velocity to a thresholdchange in angular velocity; responsive to the combined angular velocitybeing above the threshold change in angular velocity, adjusting theangular velocity of the motor of the rotating platform to reduce thedifference between the angular velocity of the camera and the targetangular velocity of the camera; and responsive to the combined angularvelocity being below the threshold change in angular velocity, adjustingthe angular velocity of the motor of the gimbal to reduce the differencebetween the angular velocity of the camera and the target angularvelocity of the camera.
 14. The non-transitory computer-readable storagemedium of claim 9, further including instructions that when executedcause the processor to perform operations including: detecting a firstangular position and a second angular position of the object relative toan orientation of the camera, the first angular position detected at afirst time and the second angular position detected at a second time;determining an angular velocity of the object based on the first angularposition, the second angular position, the first time, and the secondtime; and adjusting the angular velocity of the motor of the rotatingplatform to reduce a difference between an angular velocity of therotating platform and the angular velocity of the object.
 15. Thenon-transitory computer-readable storage medium of claim 14, furtherincluding instructions that when executed cause the processor to performoperations including: determining a target angular position of the motorof the gimbal based on an angular position of the motor of the gimbal,the second angular position of the object, and an angular position ofthe camera; and adjusting the angular position of the motor of thegimbal to reduce a difference between the angular position of the motorof the gimbal and the target angular position of the motor.
 16. A camerasystem, comprising: an imaging device; a gimbal coupled to the imagingdevice; a rotating platform coupled to the gimbal; a processor; and amemory coupled to the processor, wherein the memory includesinstructions executable by the processor to cause the processor to:determine a variance velocity between a camera velocity and a targetvelocity; compare the variance velocity to a threshold velocity;responsive to the variance velocity not exceeding the thresholdvelocity, adjust a gimbal motor velocity to reduce the variancevelocity; and responsive to the variance velocity exceeding thethreshold velocity, adjust a rotating platform motor velocity to reducethe variance velocity.
 17. The camera system of claim 16, wherein thememory further includes instructions executable by the processor tocause the processor to: responsive to the variance velocity notexceeding the threshold velocity, determine if the rotating platform isstationary; responsive to the rotating platform being stationary, adjustthe gimbal motor velocity to reduce the variance velocity; andresponsive to the rotating platform not being stationary, decelerate therotating platform and adjust the gimbal motor velocity to reduce anoffset between the gimbal motor velocity and the target velocity. 18.The camera system of claim 16, wherein the target velocity is suitablefor tracking an object.
 19. The camera system of claim 18, wherein thememory further includes instructions executable by the processor tocause the processor to: detect a first position and a second position ofthe object relative to an orientation of the camera, the first positiondetected at a first time and the second position detected at a secondtime; determine an object velocity based on the first position, thesecond position, the first time, and the second time; and adjust therotating platform motor velocity to reduce a difference between arotating platform velocity and the object velocity.
 20. The camerasystem of claim 19, wherein the memory further includes instructionsexecutable by the processor to cause the processor to: determine atarget gimbal motor position based on a gimbal motor position, thesecond position of the object, and a camera position; and adjusting thegimbal motor position to reduce a difference between the gimbal motorposition and the target gimbal motor position.